Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_no_mpnn_from_auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_no_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 5,
 'device': device(type='cuda', index=1),
 'dropout': 0.15000000000000002,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': True,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 2000,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_no_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Validation auc = 0.541315
Epoch 1
Validation auc = 0.687324
Epoch 2
Validation auc = 0.613850
Epoch 3
Validation auc = 0.745540
Epoch 4
Loss = 9.0394e+01, PNorm = 50.2786, GNorm = 2530.6328, lr_0 = 6.5000e-04
Validation auc = 0.667606
Epoch 5
Validation auc = 0.689671
Epoch 6
Validation auc = 0.687089
Epoch 7
Validation auc = 0.542254
Epoch 8
Validation auc = 0.607981
Epoch 9
Loss = 7.4605e+01, PNorm = 50.4151, GNorm = 1465.0108, lr_0 = 9.9743e-04
Validation auc = 0.503756
Epoch 10
Validation auc = 0.691549
Epoch 11
Validation auc = 0.682864
Epoch 12
Validation auc = 0.717371
Epoch 13
Validation auc = 0.679343
Epoch 14
Loss = 6.4015e+01, PNorm = 50.5723, GNorm = 1035.0876, lr_0 = 9.8890e-04
Validation auc = 0.685446
Epoch 15
Validation auc = 0.704460
Epoch 16
Validation auc = 0.697653
Epoch 17
Validation auc = 0.622535
Epoch 18
Validation auc = 0.519014
Epoch 19
Loss = 3.3068e+01, PNorm = 50.6958, GNorm = 447.1608, lr_0 = 9.8045e-04
Validation auc = 0.693427
Epoch 20
Validation auc = 0.752582
Epoch 21
Validation auc = 0.750235
Epoch 22
Validation auc = 0.703756
Epoch 23
Validation auc = 0.665258
Epoch 24
Loss = 2.4922e+01, PNorm = 50.7819, GNorm = 706.5276, lr_0 = 9.7207e-04
Validation auc = 0.653052
Epoch 25
Validation auc = 0.700939
Epoch 26
Validation auc = 0.694366
Epoch 27
Validation auc = 0.705164
Epoch 28
Validation auc = 0.599765
Epoch 29
Loss = 1.0336e+01, PNorm = 50.8403, GNorm = 206.3811, lr_0 = 9.6376e-04
Validation auc = 0.677465
Epoch 30
Validation auc = 0.714085
Epoch 31
Validation auc = 0.722066
Epoch 32
Validation auc = 0.734742
Epoch 33
Validation auc = 0.619718
Epoch 34
Loss = 9.0403e+00, PNorm = 50.8827, GNorm = 275.4954, lr_0 = 9.5552e-04
Validation auc = 0.647183
Epoch 35
Validation auc = 0.736150
Epoch 36
Validation auc = 0.738263
Epoch 37
Validation auc = 0.713380
Epoch 38
Validation auc = 0.667371
Epoch 39
Loss = 5.3459e+00, PNorm = 50.9166, GNorm = 291.9403, lr_0 = 9.4735e-04
Validation auc = 0.628638
Epoch 40
Validation auc = 0.651408
Epoch 41
Validation auc = 0.656808
Epoch 42
Validation auc = 0.734742
Epoch 43
Validation auc = 0.709624
Epoch 44
Loss = 3.7650e+00, PNorm = 50.9449, GNorm = 159.8380, lr_0 = 9.3925e-04
Validation auc = 0.704695
Epoch 45
Validation auc = 0.722300
Epoch 46
Validation auc = 0.732160
Epoch 47
Validation auc = 0.728638
Epoch 48
Validation auc = 0.719014
Epoch 49
Loss = 5.8157e+00, PNorm = 50.9690, GNorm = 99.4597, lr_0 = 9.3122e-04
Validation auc = 0.668310
Epoch 50
Validation auc = 0.631455
Epoch 51
Validation auc = 0.543192
Epoch 52
Validation auc = 0.529343
Epoch 53
Validation auc = 0.553756
Epoch 54
Loss = 2.2943e+00, PNorm = 50.9902, GNorm = 68.4920, lr_0 = 9.2326e-04
Validation auc = 0.621831
Epoch 55
Validation auc = 0.666197
Epoch 56
Validation auc = 0.658451
Epoch 57
Validation auc = 0.660329
Epoch 58
Validation auc = 0.646714
Epoch 59
Loss = 1.9600e+00, PNorm = 51.0113, GNorm = 77.3648, lr_0 = 9.1537e-04
Validation auc = 0.651174
Epoch 60
Validation auc = 0.663850
Epoch 61
Validation auc = 0.685915
Epoch 62
Validation auc = 0.698122
Epoch 63
Validation auc = 0.704930
Epoch 64
Loss = 2.1609e+00, PNorm = 51.0321, GNorm = 112.0489, lr_0 = 9.0754e-04
Validation auc = 0.699296
Epoch 65
Validation auc = 0.681455
Epoch 66
Validation auc = 0.682629
Epoch 67
Validation auc = 0.666197
Epoch 68
Validation auc = 0.642723
Epoch 69
Loss = 2.0080e+00, PNorm = 51.0522, GNorm = 185.6429, lr_0 = 8.9978e-04
Validation auc = 0.641784
Epoch 70
Validation auc = 0.606103
Epoch 71
Validation auc = 0.633803
Epoch 72
Validation auc = 0.635681
Epoch 73
Validation auc = 0.650235
Epoch 74
Loss = 1.4909e+00, PNorm = 51.0757, GNorm = 78.2445, lr_0 = 8.9209e-04
Validation auc = 0.661033
Epoch 75
Validation auc = 0.664789
Epoch 76
Validation auc = 0.675587
Epoch 77
Validation auc = 0.657277
Epoch 78
Validation auc = 0.614554
Epoch 79
Loss = 2.0783e+00, PNorm = 51.0993, GNorm = 164.5298, lr_0 = 8.8447e-04
Validation auc = 0.631455
Epoch 80
Validation auc = 0.637559
Epoch 81
Validation auc = 0.643192
Epoch 82
Validation auc = 0.659390
Epoch 83
Validation auc = 0.681221
Epoch 84
Loss = 1.1237e+00, PNorm = 51.1210, GNorm = 39.7296, lr_0 = 8.7691e-04
Validation auc = 0.677934
Epoch 85
Validation auc = 0.648357
Epoch 86
Validation auc = 0.667371
Epoch 87
Validation auc = 0.657981
Epoch 88
Validation auc = 0.652113
Epoch 89
Loss = 2.5787e+00, PNorm = 51.1412, GNorm = 151.9210, lr_0 = 8.6941e-04
Validation auc = 0.668779
Epoch 90
Validation auc = 0.672300
Epoch 91
Validation auc = 0.676056
Epoch 92
Validation auc = 0.606103
Epoch 93
Validation auc = 0.596948
Epoch 94
Loss = 2.0995e+00, PNorm = 51.1626, GNorm = 76.1511, lr_0 = 8.6198e-04
Validation auc = 0.585681
Epoch 95
Validation auc = 0.586385
Epoch 96
Validation auc = 0.561972
Epoch 97
Validation auc = 0.578404
Epoch 98
Validation auc = 0.614789
Epoch 99
Loss = 1.7488e+00, PNorm = 51.1841, GNorm = 56.2014, lr_0 = 8.5461e-04
Validation auc = 0.632864
Epoch 100
Validation auc = 0.636385
Epoch 101
Validation auc = 0.636854
Epoch 102
Validation auc = 0.619718
Epoch 103
Validation auc = 0.606573
Epoch 104
Loss = 1.4286e+00, PNorm = 51.2018, GNorm = 70.7210, lr_0 = 8.4730e-04
Validation auc = 0.626526
Epoch 105
Validation auc = 0.639202
Epoch 106
Validation auc = 0.650704
Epoch 107
Validation auc = 0.662207
Epoch 108
Validation auc = 0.669249
Epoch 109
Loss = 1.0122e+00, PNorm = 51.2169, GNorm = 66.1309, lr_0 = 8.4006e-04
Validation auc = 0.660329
Epoch 110
Validation auc = 0.658451
Epoch 111
Validation auc = 0.656573
Epoch 112
Validation auc = 0.656103
Epoch 113
Validation auc = 0.648122
Epoch 114
Loss = 9.1863e-01, PNorm = 51.2310, GNorm = 94.7822, lr_0 = 8.3288e-04
Validation auc = 0.657746
Epoch 115
Validation auc = 0.684742
Epoch 116
Validation auc = 0.679812
Epoch 117
Validation auc = 0.705164
Epoch 118
Validation auc = 0.691315
Epoch 119
Loss = 2.8347e+00, PNorm = 51.2464, GNorm = 87.9881, lr_0 = 8.2576e-04
Validation auc = 0.687089
Epoch 120
Validation auc = 0.635446
Epoch 121
Validation auc = 0.605869
Epoch 122
Validation auc = 0.617136
Epoch 123
Validation auc = 0.642019
Epoch 124
Loss = 1.3493e+00, PNorm = 51.2618, GNorm = 110.8954, lr_0 = 8.1870e-04
Validation auc = 0.665962
Epoch 125
Validation auc = 0.657746
Epoch 126
Validation auc = 0.646948
Epoch 127
Validation auc = 0.629577
Epoch 128
Validation auc = 0.580282
Epoch 129
Loss = 9.7937e-01, PNorm = 51.2764, GNorm = 65.3172, lr_0 = 8.1170e-04
Validation auc = 0.611268
Epoch 130
Validation auc = 0.610094
Epoch 131
Validation auc = 0.611972
Epoch 132
Validation auc = 0.605399
Epoch 133
Validation auc = 0.582629
Epoch 134
Loss = 7.7761e-01, PNorm = 51.2887, GNorm = 91.4780, lr_0 = 8.0476e-04
Validation auc = 0.575587
Epoch 135
Validation auc = 0.553286
Epoch 136
Validation auc = 0.540610
Epoch 137
Validation auc = 0.545540
Epoch 138
Validation auc = 0.561972
Epoch 139
Loss = 7.0340e-01, PNorm = 51.3012, GNorm = 67.9301, lr_0 = 7.9788e-04
Validation auc = 0.550235
Epoch 140
Validation auc = 0.541784
Epoch 141
Validation auc = 0.556808
Epoch 142
Validation auc = 0.566667
Epoch 143
Validation auc = 0.569014
Epoch 144
Loss = 6.4000e-01, PNorm = 51.3135, GNorm = 68.2793, lr_0 = 7.9106e-04
Validation auc = 0.572770
Epoch 145
Validation auc = 0.631221
Epoch 146
Validation auc = 0.626761
Epoch 147
Validation auc = 0.636620
Epoch 148
Validation auc = 0.653991
Epoch 149
Loss = 9.3745e-01, PNorm = 51.3260, GNorm = 90.4378, lr_0 = 7.8430e-04
Validation auc = 0.686385
Epoch 150
Validation auc = 0.691080
Epoch 151
Validation auc = 0.685915
Epoch 152
Validation auc = 0.669484
Epoch 153
Validation auc = 0.654225
Epoch 154
Loss = 8.3698e-01, PNorm = 51.3384, GNorm = 53.0681, lr_0 = 7.7759e-04
Validation auc = 0.639437
Epoch 155
Validation auc = 0.621127
Epoch 156
Validation auc = 0.620657
Epoch 157
Validation auc = 0.619484
Epoch 158
Validation auc = 0.626291
Epoch 159
Loss = 2.1275e+00, PNorm = 51.3483, GNorm = 119.4018, lr_0 = 7.7095e-04
Validation auc = 0.630516
Epoch 160
Validation auc = 0.641784
Epoch 161
Validation auc = 0.653991
Epoch 162
Validation auc = 0.658216
Epoch 163
Validation auc = 0.652582
Epoch 164
Loss = 4.3261e-01, PNorm = 51.3585, GNorm = 65.8497, lr_0 = 7.6436e-04
Validation auc = 0.646479
Epoch 165
Validation auc = 0.636150
Epoch 166
Validation auc = 0.626526
Epoch 167
Validation auc = 0.621596
Epoch 168
Validation auc = 0.622535
Epoch 169
Loss = 6.0246e-01, PNorm = 51.3688, GNorm = 41.4429, lr_0 = 7.5782e-04
Validation auc = 0.629108
Epoch 170
Validation auc = 0.636620
Epoch 171
Validation auc = 0.639437
Epoch 172
Validation auc = 0.640845
Epoch 173
Validation auc = 0.646948
Epoch 174
Loss = 2.3214e+00, PNorm = 51.3803, GNorm = 53.1235, lr_0 = 7.5134e-04
Validation auc = 0.662441
Epoch 175
Validation auc = 0.673239
Epoch 176
Validation auc = 0.687324
Epoch 177
Validation auc = 0.676995
Epoch 178
Validation auc = 0.678873
Epoch 179
Loss = 8.8907e-01, PNorm = 51.3915, GNorm = 56.4599, lr_0 = 7.4492e-04
Validation auc = 0.695775
Epoch 180
Validation auc = 0.709390
Epoch 181
Validation auc = 0.706103
Epoch 182
Validation auc = 0.685446
Epoch 183
Validation auc = 0.670892
Epoch 184
Loss = 9.1962e-01, PNorm = 51.4013, GNorm = 61.0697, lr_0 = 7.3855e-04
Validation auc = 0.694836
Epoch 185
Validation auc = 0.699061
Epoch 186
Validation auc = 0.689671
Epoch 187
Validation auc = 0.681221
Epoch 188
Validation auc = 0.680751
Epoch 189
Loss = 6.1983e-01, PNorm = 51.4103, GNorm = 32.3636, lr_0 = 7.3224e-04
Validation auc = 0.670892
Epoch 190
Validation auc = 0.666667
Epoch 191
Validation auc = 0.646479
Epoch 192
Validation auc = 0.627230
Epoch 193
Validation auc = 0.594366
Epoch 194
Loss = 9.1954e-01, PNorm = 51.4191, GNorm = 104.8413, lr_0 = 7.2598e-04
Validation auc = 0.571362
Epoch 195
Validation auc = 0.570892
Epoch 196
Validation auc = 0.603756
Epoch 197
Validation auc = 0.628169
Epoch 198
Validation auc = 0.623474
Epoch 199
Loss = 3.6545e-01, PNorm = 51.4297, GNorm = 42.8224, lr_0 = 7.1977e-04
Validation auc = 0.625352
Epoch 200
Validation auc = 0.651643
Epoch 201
Validation auc = 0.644601
Epoch 202
Validation auc = 0.622066
Epoch 203
Validation auc = 0.609859
Epoch 204
Loss = 2.0742e-01, PNorm = 51.4391, GNorm = 13.7042, lr_0 = 7.1362e-04
Validation auc = 0.606573
Epoch 205
Validation auc = 0.625822
Epoch 206
Validation auc = 0.657512
Epoch 207
Validation auc = 0.625822
Epoch 208
Validation auc = 0.620892
Epoch 209
Loss = 3.9302e-01, PNorm = 51.4482, GNorm = 32.7880, lr_0 = 7.0752e-04
Validation auc = 0.676291
Epoch 210
Validation auc = 0.687559
Epoch 211
Validation auc = 0.665493
Epoch 212
Validation auc = 0.637324
Epoch 213
Validation auc = 0.614319
Epoch 214
Loss = 3.7788e-01, PNorm = 51.4568, GNorm = 49.9992, lr_0 = 7.0147e-04
Validation auc = 0.622770
Epoch 215
Validation auc = 0.661268
Epoch 216
Validation auc = 0.666901
Epoch 217
Validation auc = 0.665962
Epoch 218
Validation auc = 0.657981
Epoch 219
Loss = 6.1386e-01, PNorm = 51.4641, GNorm = 32.5045, lr_0 = 6.9548e-04
Validation auc = 0.673239
Epoch 220
Validation auc = 0.676761
Epoch 221
Validation auc = 0.677465
Epoch 222
Validation auc = 0.679343
Epoch 223
Validation auc = 0.670657
Epoch 224
Loss = 7.2303e-01, PNorm = 51.4710, GNorm = 44.3223, lr_0 = 6.8953e-04
Validation auc = 0.652817
Epoch 225
Validation auc = 0.641549
Epoch 226
Validation auc = 0.640141
Epoch 227
Validation auc = 0.650469
Epoch 228
Validation auc = 0.659390
Epoch 229
Loss = 4.3938e-01, PNorm = 51.4792, GNorm = 29.8822, lr_0 = 6.8364e-04
Validation auc = 0.672066
Epoch 230
Validation auc = 0.687089
Epoch 231
Validation auc = 0.692958
Epoch 232
Validation auc = 0.694131
Epoch 233
Validation auc = 0.684742
Epoch 234
Loss = 8.0926e-01, PNorm = 51.4877, GNorm = 19.6256, lr_0 = 6.7779e-04
Validation auc = 0.674883
Epoch 235
Validation auc = 0.669718
Epoch 236
Validation auc = 0.666901
Epoch 237
Validation auc = 0.677230
Epoch 238
Validation auc = 0.688498
Epoch 239
Loss = 6.6489e-01, PNorm = 51.4964, GNorm = 33.2196, lr_0 = 6.7200e-04
Validation auc = 0.688028
Epoch 240
Validation auc = 0.685681
Epoch 241
Validation auc = 0.688028
Epoch 242
Validation auc = 0.692254
Epoch 243
Validation auc = 0.690376
Epoch 244
Loss = 6.5432e-01, PNorm = 51.5043, GNorm = 41.3254, lr_0 = 6.6625e-04
Validation auc = 0.687559
Epoch 245
Validation auc = 0.686150
Epoch 246
Validation auc = 0.679812
Epoch 247
Validation auc = 0.666667
Epoch 248
Validation auc = 0.648357
Epoch 249
Loss = 1.6459e-01, PNorm = 51.5118, GNorm = 14.8851, lr_0 = 6.6056e-04
Validation auc = 0.620188
Epoch 250
Validation auc = 0.617371
Epoch 251
Validation auc = 0.623474
Epoch 252
Validation auc = 0.652582
Epoch 253
Validation auc = 0.653052
Epoch 254
Loss = 3.2032e-01, PNorm = 51.5188, GNorm = 23.1965, lr_0 = 6.5491e-04
Validation auc = 0.656338
Epoch 255
Validation auc = 0.665962
Epoch 256
Validation auc = 0.667606
Epoch 257
Validation auc = 0.661972
Epoch 258
Validation auc = 0.650235
Epoch 259
Loss = 2.2280e-01, PNorm = 51.5256, GNorm = 11.0475, lr_0 = 6.4931e-04
Validation auc = 0.647418
Epoch 260
Validation auc = 0.636620
Epoch 261
Validation auc = 0.625822
Epoch 262
Validation auc = 0.624413
Epoch 263
Validation auc = 0.629108
Epoch 264
Loss = 6.3475e-01, PNorm = 51.5318, GNorm = 54.7420, lr_0 = 6.4376e-04
Validation auc = 0.631455
Epoch 265
Validation auc = 0.644601
Epoch 266
Validation auc = 0.641315
Epoch 267
Validation auc = 0.646009
Epoch 268
Validation auc = 0.661972
Epoch 269
Loss = 1.9689e-01, PNorm = 51.5378, GNorm = 13.4521, lr_0 = 6.3826e-04
Validation auc = 0.664789
Epoch 270
Validation auc = 0.671362
Epoch 271
Validation auc = 0.677934
Epoch 272
Validation auc = 0.682629
Epoch 273
Validation auc = 0.686385
Epoch 274
Loss = 1.9518e-01, PNorm = 51.5436, GNorm = 14.9883, lr_0 = 6.3280e-04
Validation auc = 0.698122
Epoch 275
Validation auc = 0.704695
Epoch 276
Validation auc = 0.707981
Epoch 277
Validation auc = 0.707042
Epoch 278
Validation auc = 0.697653
Epoch 279
Loss = 1.8394e-01, PNorm = 51.5488, GNorm = 25.8312, lr_0 = 6.2739e-04
Validation auc = 0.698592
Epoch 280
Validation auc = 0.697653
Epoch 281
Validation auc = 0.695305
Epoch 282
Validation auc = 0.698122
Epoch 283
Validation auc = 0.700000
Epoch 284
Loss = 2.6241e-01, PNorm = 51.5548, GNorm = 16.4058, lr_0 = 6.2203e-04
Validation auc = 0.704225
Epoch 285
Validation auc = 0.708451
Epoch 286
Validation auc = 0.710329
Epoch 287
Validation auc = 0.719249
Epoch 288
Validation auc = 0.730986
Epoch 289
Loss = 1.9666e-01, PNorm = 51.5612, GNorm = 20.7545, lr_0 = 6.1671e-04
Validation auc = 0.730047
Epoch 290
Validation auc = 0.722066
Epoch 291
Validation auc = 0.723005
Epoch 292
Validation auc = 0.722066
Epoch 293
Validation auc = 0.717371
Epoch 294
Loss = 3.0834e-01, PNorm = 51.5668, GNorm = 36.4549, lr_0 = 6.1144e-04
Validation auc = 0.703756
Epoch 295
Validation auc = 0.700000
Epoch 296
Validation auc = 0.701408
Epoch 297
Validation auc = 0.695305
Epoch 298
Validation auc = 0.686385
Epoch 299
Loss = 4.4940e-01, PNorm = 51.5729, GNorm = 48.3396, lr_0 = 6.0621e-04
Validation auc = 0.668075
Model 0 best validation auc = 0.752582 on epoch 20
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.670078
Ensemble test auc = 0.670078
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Validation auc = 0.664789
Epoch 1
Validation auc = 0.687324
Epoch 2
Validation auc = 0.696009
Epoch 3
Validation auc = 0.738263
Epoch 4
Loss = 4.6447e+01, PNorm = 50.2698, GNorm = 894.0388, lr_0 = 6.5000e-04
Validation auc = 0.488263
Epoch 5
Validation auc = 0.742019
Epoch 6
Validation auc = 0.635681
Epoch 7
Validation auc = 0.592019
Epoch 8
Validation auc = 0.735915
Epoch 9
Loss = 7.0623e+01, PNorm = 50.4065, GNorm = 1079.4584, lr_0 = 9.9743e-04
Validation auc = 0.652817
Epoch 10
Validation auc = 0.592723
Epoch 11
Validation auc = 0.631221
Epoch 12
Validation auc = 0.629577
Epoch 13
Validation auc = 0.569718
Epoch 14
Loss = 3.1920e+01, PNorm = 50.5651, GNorm = 624.6703, lr_0 = 9.8890e-04
Validation auc = 0.697418
Epoch 15
Validation auc = 0.719249
Epoch 16
Validation auc = 0.659155
Epoch 17
Validation auc = 0.691549
Epoch 18
Validation auc = 0.595775
Epoch 19
Loss = 2.1016e+01, PNorm = 50.6795, GNorm = 446.5110, lr_0 = 9.8045e-04
Validation auc = 0.523239
Epoch 20
Validation auc = 0.570423
Epoch 21
Validation auc = 0.736150
Epoch 22
Validation auc = 0.724413
Epoch 23
Validation auc = 0.628169
Epoch 24
Loss = 1.9335e+01, PNorm = 50.7608, GNorm = 646.4997, lr_0 = 9.7207e-04
Validation auc = 0.668545
Epoch 25
Validation auc = 0.748826
Epoch 26
Validation auc = 0.748826
Epoch 27
Validation auc = 0.650939
Epoch 28
Validation auc = 0.613850
Epoch 29
Loss = 7.3303e+00, PNorm = 50.8202, GNorm = 240.2265, lr_0 = 9.6376e-04
Validation auc = 0.669484
Epoch 30
Validation auc = 0.677465
Epoch 31
Validation auc = 0.673709
Epoch 32
Validation auc = 0.692958
Epoch 33
Validation auc = 0.678638
Epoch 34
Loss = 8.9368e+00, PNorm = 50.8626, GNorm = 171.2612, lr_0 = 9.5552e-04
Validation auc = 0.691549
Epoch 35
Validation auc = 0.688263
Epoch 36
Validation auc = 0.696479
Epoch 37
Validation auc = 0.692723
Epoch 38
Validation auc = 0.718075
Epoch 39
Loss = 8.7747e+00, PNorm = 50.8950, GNorm = 118.6954, lr_0 = 9.4735e-04
Validation auc = 0.712207
Epoch 40
Validation auc = 0.731690
Epoch 41
Validation auc = 0.656103
Epoch 42
Validation auc = 0.703521
Epoch 43
Validation auc = 0.727934
Epoch 44
Loss = 5.1977e+00, PNorm = 50.9248, GNorm = 348.3760, lr_0 = 9.3925e-04
Validation auc = 0.732629
Epoch 45
Validation auc = 0.685915
Epoch 46
Validation auc = 0.667136
Epoch 47
Validation auc = 0.687559
Epoch 48
Validation auc = 0.681221
Epoch 49
Loss = 6.1086e+00, PNorm = 50.9523, GNorm = 228.7707, lr_0 = 9.3122e-04
Validation auc = 0.650469
Epoch 50
Validation auc = 0.673709
Epoch 51
Validation auc = 0.701643
Epoch 52
Validation auc = 0.692958
Epoch 53
Validation auc = 0.695775
Epoch 54
Loss = 5.7918e+00, PNorm = 50.9777, GNorm = 231.4018, lr_0 = 9.2326e-04
Validation auc = 0.688967
Epoch 55
Validation auc = 0.696009
Epoch 56
Validation auc = 0.693662
Epoch 57
Validation auc = 0.661972
Epoch 58
Validation auc = 0.601408
Epoch 59
Loss = 1.1742e+00, PNorm = 50.9994, GNorm = 98.6379, lr_0 = 9.1537e-04
Validation auc = 0.601174
Epoch 60
Validation auc = 0.635211
Epoch 61
Validation auc = 0.642254
Epoch 62
Validation auc = 0.612441
Epoch 63
Validation auc = 0.647183
Epoch 64
Loss = 1.9113e+00, PNorm = 51.0194, GNorm = 103.9640, lr_0 = 9.0754e-04
Validation auc = 0.662676
Epoch 65
Validation auc = 0.645540
Epoch 66
Validation auc = 0.632629
Epoch 67
Validation auc = 0.666197
Epoch 68
Validation auc = 0.658920
Epoch 69
Loss = 1.0153e+00, PNorm = 51.0403, GNorm = 95.2792, lr_0 = 8.9978e-04
Validation auc = 0.652582
Epoch 70
Validation auc = 0.705634
Epoch 71
Validation auc = 0.711972
Epoch 72
Validation auc = 0.698592
Epoch 73
Validation auc = 0.713850
Epoch 74
Loss = 3.0693e+00, PNorm = 51.0627, GNorm = 205.9186, lr_0 = 8.9209e-04
Validation auc = 0.749765
Epoch 75
Validation auc = 0.733333
Epoch 76
Validation auc = 0.703052
Epoch 77
Validation auc = 0.674648
Epoch 78
Validation auc = 0.681925
Epoch 79
Loss = 3.7844e+00, PNorm = 51.0877, GNorm = 155.5273, lr_0 = 8.8447e-04
Validation auc = 0.671127
Epoch 80
Validation auc = 0.675822
Epoch 81
Validation auc = 0.642723
Epoch 82
Validation auc = 0.644601
Epoch 83
Validation auc = 0.700235
Epoch 84
Loss = 1.1716e+00, PNorm = 51.1119, GNorm = 99.3948, lr_0 = 8.7691e-04
Validation auc = 0.693427
Epoch 85
Validation auc = 0.688498
Epoch 86
Validation auc = 0.701643
Epoch 87
Validation auc = 0.646479
Epoch 88
Validation auc = 0.622066
Epoch 89
Loss = 3.4237e+00, PNorm = 51.1345, GNorm = 146.7545, lr_0 = 8.6941e-04
Validation auc = 0.595305
Epoch 90
Validation auc = 0.564085
Epoch 91
Validation auc = 0.677934
Epoch 92
Validation auc = 0.669718
Epoch 93
Validation auc = 0.650939
Epoch 94
Loss = 1.6969e+00, PNorm = 51.1558, GNorm = 133.6981, lr_0 = 8.6198e-04
Validation auc = 0.654225
Epoch 95
Validation auc = 0.680516
Epoch 96
Validation auc = 0.673239
Epoch 97
Validation auc = 0.684272
Epoch 98
Validation auc = 0.674648
Epoch 99
Loss = 1.0923e+00, PNorm = 51.1757, GNorm = 126.8637, lr_0 = 8.5461e-04
Validation auc = 0.675822
Epoch 100
Validation auc = 0.694131
Epoch 101
Validation auc = 0.694836
Epoch 102
Validation auc = 0.659390
Epoch 103
Validation auc = 0.658685
Epoch 104
Loss = 1.1037e+00, PNorm = 51.1923, GNorm = 65.9528, lr_0 = 8.4730e-04
Validation auc = 0.686620
Epoch 105
Validation auc = 0.702113
Epoch 106
Validation auc = 0.707981
Epoch 107
Validation auc = 0.716667
Epoch 108
Validation auc = 0.718545
Epoch 109
Loss = 7.7193e-01, PNorm = 51.2077, GNorm = 32.3957, lr_0 = 8.4006e-04
Validation auc = 0.715962
Epoch 110
Validation auc = 0.706338
Epoch 111
Validation auc = 0.693662
Epoch 112
Validation auc = 0.684507
Epoch 113
Validation auc = 0.648826
Epoch 114
Loss = 1.6420e+00, PNorm = 51.2203, GNorm = 102.4120, lr_0 = 8.3288e-04
Validation auc = 0.648357
Epoch 115
Validation auc = 0.654225
Epoch 116
Validation auc = 0.688028
Epoch 117
Validation auc = 0.715493
Epoch 118
Validation auc = 0.716197
Epoch 119
Loss = 1.2640e+00, PNorm = 51.2333, GNorm = 56.0446, lr_0 = 8.2576e-04
Validation auc = 0.722066
Epoch 120
Validation auc = 0.724178
Epoch 121
Validation auc = 0.715023
Epoch 122
Validation auc = 0.703286
Epoch 123
Validation auc = 0.694366
Epoch 124
Loss = 1.1042e+00, PNorm = 51.2468, GNorm = 58.8506, lr_0 = 8.1870e-04
Validation auc = 0.673709
Epoch 125
Validation auc = 0.670892
Epoch 126
Validation auc = 0.658685
Epoch 127
Validation auc = 0.650939
Epoch 128
Validation auc = 0.639906
Epoch 129
Loss = 1.1196e+00, PNorm = 51.2600, GNorm = 78.3300, lr_0 = 8.1170e-04
Validation auc = 0.664554
Epoch 130
Validation auc = 0.656573
Epoch 131
Validation auc = 0.675117
Epoch 132
Validation auc = 0.684507
Epoch 133
Validation auc = 0.709624
Epoch 134
Loss = 1.5966e+00, PNorm = 51.2724, GNorm = 214.2753, lr_0 = 8.0476e-04
Validation auc = 0.713380
Epoch 135
Validation auc = 0.690141
Epoch 136
Validation auc = 0.678873
Epoch 137
Validation auc = 0.691784
Epoch 138
Validation auc = 0.698592
Epoch 139
Loss = 4.0378e-01, PNorm = 51.2853, GNorm = 105.9368, lr_0 = 7.9788e-04
Validation auc = 0.688732
Epoch 140
Validation auc = 0.680516
Epoch 141
Validation auc = 0.690141
Epoch 142
Validation auc = 0.696009
Epoch 143
Validation auc = 0.715728
Epoch 144
Loss = 7.6953e-01, PNorm = 51.2968, GNorm = 41.5458, lr_0 = 7.9106e-04
Validation auc = 0.713615
Epoch 145
Validation auc = 0.720423
Epoch 146
Validation auc = 0.727465
Epoch 147
Validation auc = 0.723944
Epoch 148
Validation auc = 0.714554
Epoch 149
Loss = 1.2164e+00, PNorm = 51.3067, GNorm = 53.3153, lr_0 = 7.8430e-04
Validation auc = 0.692019
Epoch 150
Validation auc = 0.673944
Epoch 151
Validation auc = 0.671127
Epoch 152
Validation auc = 0.684742
Epoch 153
Validation auc = 0.655869
Epoch 154
Loss = 1.2452e+00, PNorm = 51.3162, GNorm = 82.9044, lr_0 = 7.7759e-04
Validation auc = 0.685681
Epoch 155
Validation auc = 0.672300
Epoch 156
Validation auc = 0.667136
Epoch 157
Validation auc = 0.703521
Epoch 158
Validation auc = 0.727700
Epoch 159
Loss = 1.1630e+00, PNorm = 51.3277, GNorm = 73.8947, lr_0 = 7.7095e-04
Validation auc = 0.739906
Epoch 160
Validation auc = 0.739906
Epoch 161
Validation auc = 0.725822
Epoch 162
Validation auc = 0.734038
Epoch 163
Validation auc = 0.729343
Epoch 164
Loss = 7.8277e-01, PNorm = 51.3391, GNorm = 66.4950, lr_0 = 7.6436e-04
Validation auc = 0.725117
Epoch 165
Validation auc = 0.717136
Epoch 166
Validation auc = 0.690376
Epoch 167
Validation auc = 0.657746
Epoch 168
Validation auc = 0.675117
Epoch 169
Loss = 6.7947e-01, PNorm = 51.3492, GNorm = 66.8179, lr_0 = 7.5782e-04
Validation auc = 0.674178
Epoch 170
Validation auc = 0.628638
Epoch 171
Validation auc = 0.678169
Epoch 172
Validation auc = 0.708920
Epoch 173
Validation auc = 0.714085
Epoch 174
Loss = 8.1817e-01, PNorm = 51.3607, GNorm = 55.6467, lr_0 = 7.5134e-04
Validation auc = 0.702817
Epoch 175
Validation auc = 0.695775
Epoch 176
Validation auc = 0.686854
Epoch 177
Validation auc = 0.678404
Epoch 178
Validation auc = 0.676056
Epoch 179
Loss = 2.4547e-01, PNorm = 51.3727, GNorm = 34.1585, lr_0 = 7.4492e-04
Validation auc = 0.675822
Epoch 180
Validation auc = 0.674413
Epoch 181
Validation auc = 0.680986
Epoch 182
Validation auc = 0.679108
Epoch 183
Validation auc = 0.661972
Epoch 184
Loss = 7.1705e-01, PNorm = 51.3849, GNorm = 66.5881, lr_0 = 7.3855e-04
Validation auc = 0.637559
Epoch 185
Validation auc = 0.623944
Epoch 186
Validation auc = 0.636854
Epoch 187
Validation auc = 0.655634
Epoch 188
Validation auc = 0.654460
Epoch 189
Loss = 8.1447e-01, PNorm = 51.3955, GNorm = 53.0389, lr_0 = 7.3224e-04
Validation auc = 0.649296
Epoch 190
Validation auc = 0.634742
Epoch 191
Validation auc = 0.616432
Epoch 192
Validation auc = 0.599531
Epoch 193
Validation auc = 0.598592
Epoch 194
Loss = 6.2955e-01, PNorm = 51.4071, GNorm = 58.4532, lr_0 = 7.2598e-04
Validation auc = 0.617371
Epoch 195
Validation auc = 0.672300
Epoch 196
Validation auc = 0.718779
Epoch 197
Validation auc = 0.720657
Epoch 198
Validation auc = 0.727230
Epoch 199
Loss = 5.8914e-01, PNorm = 51.4180, GNorm = 23.0038, lr_0 = 7.1977e-04
Validation auc = 0.738498
Epoch 200
Validation auc = 0.738028
Epoch 201
Validation auc = 0.737089
Epoch 202
Validation auc = 0.727700
Epoch 203
Validation auc = 0.711502
Epoch 204
Loss = 7.3724e-01, PNorm = 51.4288, GNorm = 40.5623, lr_0 = 7.1362e-04
Validation auc = 0.695540
Epoch 205
Validation auc = 0.684038
Epoch 206
Validation auc = 0.676526
Epoch 207
Validation auc = 0.680751
Epoch 208
Validation auc = 0.671362
Epoch 209
Loss = 7.1894e-01, PNorm = 51.4381, GNorm = 22.1573, lr_0 = 7.0752e-04
Validation auc = 0.670423
Epoch 210
Validation auc = 0.691549
Epoch 211
Validation auc = 0.694366
Epoch 212
Validation auc = 0.698592
Epoch 213
Validation auc = 0.678873
Epoch 214
Loss = 3.0272e-01, PNorm = 51.4453, GNorm = 29.2349, lr_0 = 7.0147e-04
Validation auc = 0.616901
Epoch 215
Validation auc = 0.592019
Epoch 216
Validation auc = 0.585446
Epoch 217
Validation auc = 0.587793
Epoch 218
Validation auc = 0.619249
Epoch 219
Loss = 3.8631e-01, PNorm = 51.4528, GNorm = 22.5261, lr_0 = 6.9548e-04
Validation auc = 0.638967
Epoch 220
Validation auc = 0.648826
Epoch 221
Validation auc = 0.657277
Epoch 222
Validation auc = 0.658216
Epoch 223
Validation auc = 0.669014
Epoch 224
Loss = 7.1155e-01, PNorm = 51.4606, GNorm = 27.5206, lr_0 = 6.8953e-04
Validation auc = 0.663850
Epoch 225
Validation auc = 0.648357
Epoch 226
Validation auc = 0.638498
Epoch 227
Validation auc = 0.649765
Epoch 228
Validation auc = 0.656338
Epoch 229
Loss = 1.8635e-01, PNorm = 51.4679, GNorm = 20.0833, lr_0 = 6.8364e-04
Validation auc = 0.664319
Epoch 230
Validation auc = 0.680282
Epoch 231
Validation auc = 0.671831
Epoch 232
Validation auc = 0.676995
Epoch 233
Validation auc = 0.683099
Epoch 234
Loss = 5.3761e-01, PNorm = 51.4760, GNorm = 28.1212, lr_0 = 6.7779e-04
Validation auc = 0.695775
Epoch 235
Validation auc = 0.703756
Epoch 236
Validation auc = 0.699531
Epoch 237
Validation auc = 0.674648
Epoch 238
Validation auc = 0.649296
Epoch 239
Loss = 3.1875e-01, PNorm = 51.4838, GNorm = 45.8182, lr_0 = 6.7200e-04
Validation auc = 0.624883
Epoch 240
Validation auc = 0.614085
Epoch 241
Validation auc = 0.626761
Epoch 242
Validation auc = 0.638498
Epoch 243
Validation auc = 0.653052
Epoch 244
Loss = 8.0968e-01, PNorm = 51.4912, GNorm = 34.8786, lr_0 = 6.6625e-04
Validation auc = 0.641784
Epoch 245
Validation auc = 0.641784
Epoch 246
Validation auc = 0.637089
Epoch 247
Validation auc = 0.632864
Epoch 248
Validation auc = 0.623944
Epoch 249
Loss = 5.7826e-01, PNorm = 51.4980, GNorm = 20.8771, lr_0 = 6.6056e-04
Validation auc = 0.653521
Epoch 250
Validation auc = 0.663380
Epoch 251
Validation auc = 0.675587
Epoch 252
Validation auc = 0.681221
Epoch 253
Validation auc = 0.684977
Epoch 254
Loss = 3.3144e-01, PNorm = 51.5056, GNorm = 42.9770, lr_0 = 6.5491e-04
Validation auc = 0.702817
Epoch 255
Validation auc = 0.698592
Epoch 256
Validation auc = 0.691549
Epoch 257
Validation auc = 0.681221
Epoch 258
Validation auc = 0.669953
Epoch 259
Loss = 7.8917e-01, PNorm = 51.5127, GNorm = 52.4606, lr_0 = 6.4931e-04
Validation auc = 0.663380
Epoch 260
Validation auc = 0.637089
Epoch 261
Validation auc = 0.646948
Epoch 262
Validation auc = 0.676526
Epoch 263
Validation auc = 0.704695
Epoch 264
Loss = 4.6556e-01, PNorm = 51.5191, GNorm = 21.5417, lr_0 = 6.4376e-04
Validation auc = 0.726761
Epoch 265
Validation auc = 0.727230
Epoch 266
Validation auc = 0.722066
Epoch 267
Validation auc = 0.715023
Epoch 268
Validation auc = 0.705634
Epoch 269
Loss = 2.0054e-01, PNorm = 51.5263, GNorm = 10.3616, lr_0 = 6.3826e-04
Validation auc = 0.704225
Epoch 270
Validation auc = 0.696244
Epoch 271
Validation auc = 0.686854
Epoch 272
Validation auc = 0.662441
Epoch 273
Validation auc = 0.650704
Epoch 274
Loss = 5.8110e-01, PNorm = 51.5326, GNorm = 30.2579, lr_0 = 6.3280e-04
Validation auc = 0.637089
Epoch 275
Validation auc = 0.628638
Epoch 276
Validation auc = 0.638498
Epoch 277
Validation auc = 0.643192
Epoch 278
Validation auc = 0.647887
Epoch 279
Loss = 3.4755e-01, PNorm = 51.5392, GNorm = 21.7599, lr_0 = 6.2739e-04
Validation auc = 0.649765
Epoch 280
Validation auc = 0.665258
Epoch 281
Validation auc = 0.678404
Epoch 282
Validation auc = 0.684977
Epoch 283
Validation auc = 0.683568
Epoch 284
Loss = 5.6612e-01, PNorm = 51.5460, GNorm = 50.1556, lr_0 = 6.2203e-04
Validation auc = 0.687324
Epoch 285
Validation auc = 0.694366
Epoch 286
Validation auc = 0.689671
Epoch 287
Validation auc = 0.676995
Epoch 288
Validation auc = 0.661502
Epoch 289
Loss = 2.0103e-01, PNorm = 51.5535, GNorm = 17.8065, lr_0 = 6.1671e-04
Validation auc = 0.651174
Epoch 290
Validation auc = 0.644131
Epoch 291
Validation auc = 0.648357
Epoch 292
Validation auc = 0.653052
Epoch 293
Validation auc = 0.659624
Epoch 294
Loss = 7.4464e-01, PNorm = 51.5604, GNorm = 29.2875, lr_0 = 6.1144e-04
Validation auc = 0.692958
Epoch 295
Validation auc = 0.700469
Epoch 296
Validation auc = 0.692488
Epoch 297
Validation auc = 0.700939
Epoch 298
Validation auc = 0.715962
Epoch 299
Loss = 1.7569e-01, PNorm = 51.5674, GNorm = 10.7491, lr_0 = 6.0621e-04
Validation auc = 0.726761
Model 0 best validation auc = 0.749765 on epoch 74
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.707706
Ensemble test auc = 0.707706
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Validation auc = 0.569014
Epoch 1
Validation auc = 0.580047
Epoch 2
Validation auc = 0.637089
Epoch 3
Validation auc = 0.743192
Epoch 4
Loss = 9.7759e+01, PNorm = 50.2717, GNorm = 1196.9217, lr_0 = 6.5000e-04
Validation auc = 0.631455
Epoch 5
Validation auc = 0.605634
Epoch 6
Validation auc = 0.742254
Epoch 7
Validation auc = 0.718545
Epoch 8
Validation auc = 0.737324
Epoch 9
Loss = 6.3347e+01, PNorm = 50.3983, GNorm = 1996.2549, lr_0 = 9.9743e-04
Validation auc = 0.723005
Epoch 10
Validation auc = 0.607512
Epoch 11
Validation auc = 0.660798
Epoch 12
Validation auc = 0.609859
Epoch 13
Validation auc = 0.732160
Epoch 14
Loss = 4.0457e+01, PNorm = 50.5400, GNorm = 1400.0757, lr_0 = 9.8890e-04
Validation auc = 0.711972
Epoch 15
Validation auc = 0.667606
Epoch 16
Validation auc = 0.712911
Epoch 17
Validation auc = 0.735446
Epoch 18
Validation auc = 0.658451
Epoch 19
Loss = 2.2924e+01, PNorm = 50.6435, GNorm = 976.9094, lr_0 = 9.8045e-04
Validation auc = 0.644836
Epoch 20
Validation auc = 0.729577
Epoch 21
Validation auc = 0.704695
Epoch 22
Validation auc = 0.714085
Epoch 23
Validation auc = 0.748592
Epoch 24
Loss = 9.5345e+00, PNorm = 50.7106, GNorm = 555.4891, lr_0 = 9.7207e-04
Validation auc = 0.712676
Epoch 25
Validation auc = 0.671596
Epoch 26
Validation auc = 0.738028
Epoch 27
Validation auc = 0.689437
Epoch 28
Validation auc = 0.659859
Epoch 29
Loss = 2.3029e+01, PNorm = 50.7609, GNorm = 824.5486, lr_0 = 9.6376e-04
Validation auc = 0.716667
Epoch 30
Validation auc = 0.638967
Epoch 31
Validation auc = 0.740845
Epoch 32
Validation auc = 0.733568
Epoch 33
Validation auc = 0.649296
Epoch 34
Loss = 1.7903e+01, PNorm = 50.8056, GNorm = 741.8304, lr_0 = 9.5552e-04
Validation auc = 0.747418
Epoch 35
Validation auc = 0.713146
Epoch 36
Validation auc = 0.695305
Epoch 37
Validation auc = 0.651408
Epoch 38
Validation auc = 0.746479
Epoch 39
Loss = 1.1369e+01, PNorm = 50.8482, GNorm = 227.0705, lr_0 = 9.4735e-04
Validation auc = 0.773239
Epoch 40
Validation auc = 0.752113
Epoch 41
Validation auc = 0.732394
Epoch 42
Validation auc = 0.700469
Epoch 43
Validation auc = 0.580751
Epoch 44
Loss = 1.5464e+01, PNorm = 50.8821, GNorm = 258.4655, lr_0 = 9.3925e-04
Validation auc = 0.697183
Epoch 45
Validation auc = 0.601174
Epoch 46
Validation auc = 0.598826
Epoch 47
Validation auc = 0.613850
Epoch 48
Validation auc = 0.646479
Epoch 49
Loss = 1.2124e+01, PNorm = 50.9127, GNorm = 230.0170, lr_0 = 9.3122e-04
Validation auc = 0.642488
Epoch 50
Validation auc = 0.609859
Epoch 51
Validation auc = 0.693427
Epoch 52
Validation auc = 0.674883
Epoch 53
Validation auc = 0.634742
Epoch 54
Loss = 4.9175e+00, PNorm = 50.9426, GNorm = 313.0924, lr_0 = 9.2326e-04
Validation auc = 0.546948
Epoch 55
Validation auc = 0.680516
Epoch 56
Validation auc = 0.673005
Epoch 57
Validation auc = 0.695305
Epoch 58
Validation auc = 0.657042
Epoch 59
Loss = 8.0041e+00, PNorm = 50.9704, GNorm = 147.6602, lr_0 = 9.1537e-04
Validation auc = 0.710094
Epoch 60
Validation auc = 0.693427
Epoch 61
Validation auc = 0.671596
Epoch 62
Validation auc = 0.686150
Epoch 63
Validation auc = 0.627465
Epoch 64
Loss = 2.0450e+00, PNorm = 50.9973, GNorm = 112.3417, lr_0 = 9.0754e-04
Validation auc = 0.616432
Epoch 65
Validation auc = 0.594601
Epoch 66
Validation auc = 0.610798
Epoch 67
Validation auc = 0.643192
Epoch 68
Validation auc = 0.688263
Epoch 69
Loss = 3.2051e+00, PNorm = 51.0218, GNorm = 111.1858, lr_0 = 8.9978e-04
Validation auc = 0.702817
Epoch 70
Validation auc = 0.732394
Epoch 71
Validation auc = 0.719484
Epoch 72
Validation auc = 0.716667
Epoch 73
Validation auc = 0.729343
Epoch 74
Loss = 2.0038e+00, PNorm = 51.0441, GNorm = 172.4180, lr_0 = 8.9209e-04
Validation auc = 0.729577
Epoch 75
Validation auc = 0.713850
Epoch 76
Validation auc = 0.702582
Epoch 77
Validation auc = 0.714554
Epoch 78
Validation auc = 0.702582
Epoch 79
Loss = 2.9355e+00, PNorm = 51.0634, GNorm = 166.1856, lr_0 = 8.8447e-04
Validation auc = 0.684272
Epoch 80
Validation auc = 0.643192
Epoch 81
Validation auc = 0.649061
Epoch 82
Validation auc = 0.694131
Epoch 83
Validation auc = 0.689202
Epoch 84
Loss = 2.8573e+00, PNorm = 51.0796, GNorm = 117.5029, lr_0 = 8.7691e-04
Validation auc = 0.676526
Epoch 85
Validation auc = 0.647653
Epoch 86
Validation auc = 0.656103
Epoch 87
Validation auc = 0.669014
Epoch 88
Validation auc = 0.626995
Epoch 89
Loss = 2.5554e+00, PNorm = 51.0953, GNorm = 163.8635, lr_0 = 8.6941e-04
Validation auc = 0.623239
Epoch 90
Validation auc = 0.632394
Epoch 91
Validation auc = 0.658920
Epoch 92
Validation auc = 0.664789
Epoch 93
Validation auc = 0.706103
Epoch 94
Loss = 2.0927e+00, PNorm = 51.1132, GNorm = 73.4927, lr_0 = 8.6198e-04
Validation auc = 0.696948
Epoch 95
Validation auc = 0.680986
Epoch 96
Validation auc = 0.650704
Epoch 97
Validation auc = 0.642958
Epoch 98
Validation auc = 0.608920
Epoch 99
Loss = 1.3428e+00, PNorm = 51.1299, GNorm = 93.4642, lr_0 = 8.5461e-04
Validation auc = 0.589202
Epoch 100
Validation auc = 0.581455
Epoch 101
Validation auc = 0.608216
Epoch 102
Validation auc = 0.644366
Epoch 103
Validation auc = 0.638967
Epoch 104
Loss = 2.4825e+00, PNorm = 51.1454, GNorm = 85.5167, lr_0 = 8.4730e-04
Validation auc = 0.677230
Epoch 105
Validation auc = 0.708216
Epoch 106
Validation auc = 0.697653
Epoch 107
Validation auc = 0.662676
Epoch 108
Validation auc = 0.646948
Epoch 109
Loss = 2.4441e+00, PNorm = 51.1615, GNorm = 179.3990, lr_0 = 8.4006e-04
Validation auc = 0.673474
Epoch 110
Validation auc = 0.669249
Epoch 111
Validation auc = 0.676995
Epoch 112
Validation auc = 0.672066
Epoch 113
Validation auc = 0.648592
Epoch 114
Loss = 2.3028e+00, PNorm = 51.1770, GNorm = 74.2502, lr_0 = 8.3288e-04
Validation auc = 0.648122
Epoch 115
Validation auc = 0.646244
Epoch 116
Validation auc = 0.651174
Epoch 117
Validation auc = 0.666667
Epoch 118
Validation auc = 0.689202
Epoch 119
Loss = 5.8595e-01, PNorm = 51.1937, GNorm = 57.3236, lr_0 = 8.2576e-04
Validation auc = 0.702582
Epoch 120
Validation auc = 0.702817
Epoch 121
Validation auc = 0.722770
Epoch 122
Validation auc = 0.724648
Epoch 123
Validation auc = 0.728873
Epoch 124
Loss = 6.5841e-01, PNorm = 51.2086, GNorm = 48.9106, lr_0 = 8.1870e-04
Validation auc = 0.716901
Epoch 125
Validation auc = 0.719718
Epoch 126
Validation auc = 0.722300
Epoch 127
Validation auc = 0.709390
Epoch 128
Validation auc = 0.692488
Epoch 129
Loss = 1.2963e+00, PNorm = 51.2206, GNorm = 172.9485, lr_0 = 8.1170e-04
Validation auc = 0.715023
Epoch 130
Validation auc = 0.730047
Epoch 131
Validation auc = 0.738732
Epoch 132
Validation auc = 0.729343
Epoch 133
Validation auc = 0.718779
Epoch 134
Loss = 3.8348e-01, PNorm = 51.2318, GNorm = 63.2518, lr_0 = 8.0476e-04
Validation auc = 0.703756
Epoch 135
Validation auc = 0.703286
Epoch 136
Validation auc = 0.703991
Epoch 137
Validation auc = 0.716901
Epoch 138
Validation auc = 0.703052
Epoch 139
Loss = 8.8640e-01, PNorm = 51.2429, GNorm = 82.3996, lr_0 = 7.9788e-04
Validation auc = 0.678873
Epoch 140
Validation auc = 0.663615
Epoch 141
Validation auc = 0.646244
Epoch 142
Validation auc = 0.661972
Epoch 143
Validation auc = 0.671127
Epoch 144
Loss = 1.9216e+00, PNorm = 51.2539, GNorm = 98.2543, lr_0 = 7.9106e-04
Validation auc = 0.633803
Epoch 145
Validation auc = 0.610798
Epoch 146
Validation auc = 0.645070
Epoch 147
Validation auc = 0.659155
Epoch 148
Validation auc = 0.670423
Epoch 149
Loss = 1.2658e+00, PNorm = 51.2647, GNorm = 44.7003, lr_0 = 7.8430e-04
Validation auc = 0.691080
Epoch 150
Validation auc = 0.708920
Epoch 151
Validation auc = 0.718310
Epoch 152
Validation auc = 0.701408
Epoch 153
Validation auc = 0.682160
Epoch 154
Loss = 8.7718e-01, PNorm = 51.2749, GNorm = 86.7585, lr_0 = 7.7759e-04
Validation auc = 0.684977
Epoch 155
Validation auc = 0.682160
Epoch 156
Validation auc = 0.673709
Epoch 157
Validation auc = 0.668075
Epoch 158
Validation auc = 0.664789
Epoch 159
Loss = 9.7889e-01, PNorm = 51.2869, GNorm = 44.6635, lr_0 = 7.7095e-04
Validation auc = 0.661972
Epoch 160
Validation auc = 0.653521
Epoch 161
Validation auc = 0.633803
Epoch 162
Validation auc = 0.585446
Epoch 163
Validation auc = 0.578404
Epoch 164
Loss = 5.9671e-01, PNorm = 51.2994, GNorm = 63.6601, lr_0 = 7.6436e-04
Validation auc = 0.628638
Epoch 165
Validation auc = 0.666197
Epoch 166
Validation auc = 0.677934
Epoch 167
Validation auc = 0.692958
Epoch 168
Validation auc = 0.685915
Epoch 169
Loss = 4.0112e-01, PNorm = 51.3108, GNorm = 43.3918, lr_0 = 7.5782e-04
Validation auc = 0.672535
Epoch 170
Validation auc = 0.656573
Epoch 171
Validation auc = 0.646948
Epoch 172
Validation auc = 0.629108
Epoch 173
Validation auc = 0.653991
Epoch 174
Loss = 7.9526e-01, PNorm = 51.3204, GNorm = 58.1873, lr_0 = 7.5134e-04
Validation auc = 0.678169
Epoch 175
Validation auc = 0.706338
Epoch 176
Validation auc = 0.684038
Epoch 177
Validation auc = 0.642488
Epoch 178
Validation auc = 0.650704
Epoch 179
Loss = 4.4566e-01, PNorm = 51.3292, GNorm = 49.1637, lr_0 = 7.4492e-04
Validation auc = 0.692019
Epoch 180
Validation auc = 0.698592
Epoch 181
Validation auc = 0.707746
Epoch 182
Validation auc = 0.713380
Epoch 183
Validation auc = 0.718545
Epoch 184
Loss = 1.3211e+00, PNorm = 51.3361, GNorm = 29.8979, lr_0 = 7.3855e-04
Validation auc = 0.725352
Epoch 185
Validation auc = 0.712676
Epoch 186
Validation auc = 0.705634
Epoch 187
Validation auc = 0.695775
Epoch 188
Validation auc = 0.700000
Epoch 189
Loss = 6.8045e-01, PNorm = 51.3422, GNorm = 59.5368, lr_0 = 7.3224e-04
Validation auc = 0.688263
Epoch 190
Validation auc = 0.665493
Epoch 191
Validation auc = 0.673944
Epoch 192
Validation auc = 0.661502
Epoch 193
Validation auc = 0.649765
Epoch 194
Loss = 1.0257e+00, PNorm = 51.3485, GNorm = 82.3665, lr_0 = 7.2598e-04
Validation auc = 0.644131
Epoch 195
Validation auc = 0.664789
Epoch 196
Validation auc = 0.661737
Epoch 197
Validation auc = 0.669014
Epoch 198
Validation auc = 0.671831
Epoch 199
Loss = 4.9375e-01, PNorm = 51.3557, GNorm = 35.1210, lr_0 = 7.1977e-04
Validation auc = 0.682160
Epoch 200
Validation auc = 0.682160
Epoch 201
Validation auc = 0.684507
Epoch 202
Validation auc = 0.684742
Epoch 203
Validation auc = 0.696479
Epoch 204
Loss = 2.6987e-01, PNorm = 51.3629, GNorm = 36.1222, lr_0 = 7.1362e-04
Validation auc = 0.706103
Epoch 205
Validation auc = 0.711268
Epoch 206
Validation auc = 0.710798
Epoch 207
Validation auc = 0.708920
Epoch 208
Validation auc = 0.708451
Epoch 209
Loss = 1.7223e-01, PNorm = 51.3704, GNorm = 24.4379, lr_0 = 7.0752e-04
Validation auc = 0.704695
Epoch 210
Validation auc = 0.689202
Epoch 211
Validation auc = 0.669953
Epoch 212
Validation auc = 0.645540
Epoch 213
Validation auc = 0.654460
Epoch 214
Loss = 3.5834e-01, PNorm = 51.3779, GNorm = 24.4072, lr_0 = 7.0147e-04
Validation auc = 0.681690
Epoch 215
Validation auc = 0.696714
Epoch 216
Validation auc = 0.703756
Epoch 217
Validation auc = 0.717840
Epoch 218
Validation auc = 0.718779
Epoch 219
Loss = 2.1589e-01, PNorm = 51.3851, GNorm = 27.8313, lr_0 = 6.9548e-04
Validation auc = 0.709859
Epoch 220
Validation auc = 0.709390
Epoch 221
Validation auc = 0.716432
Epoch 222
Validation auc = 0.726761
Epoch 223
Validation auc = 0.721127
Epoch 224
Loss = 3.6978e-01, PNorm = 51.3922, GNorm = 32.7615, lr_0 = 6.8953e-04
Validation auc = 0.693897
Epoch 225
Validation auc = 0.671362
Epoch 226
Validation auc = 0.663850
Epoch 227
Validation auc = 0.666197
Epoch 228
Validation auc = 0.677465
Epoch 229
Loss = 3.7585e-01, PNorm = 51.3995, GNorm = 31.2115, lr_0 = 6.8364e-04
Validation auc = 0.687324
Epoch 230
Validation auc = 0.693897
Epoch 231
Validation auc = 0.705634
Epoch 232
Validation auc = 0.710798
Epoch 233
Validation auc = 0.714554
Epoch 234
Loss = 2.0755e-01, PNorm = 51.4068, GNorm = 61.9002, lr_0 = 6.7779e-04
Validation auc = 0.723474
Epoch 235
Validation auc = 0.729577
Epoch 236
Validation auc = 0.733803
Epoch 237
Validation auc = 0.738967
Epoch 238
Validation auc = 0.750469
Epoch 239
Loss = 4.8355e-01, PNorm = 51.4138, GNorm = 54.4555, lr_0 = 6.7200e-04
Validation auc = 0.712911
Epoch 240
Validation auc = 0.723709
Epoch 241
Validation auc = 0.710563
Epoch 242
Validation auc = 0.696714
Epoch 243
Validation auc = 0.691080
Epoch 244
Loss = 8.7707e-01, PNorm = 51.4227, GNorm = 50.5202, lr_0 = 6.6625e-04
Validation auc = 0.695775
Epoch 245
Validation auc = 0.705634
Epoch 246
Validation auc = 0.674648
Epoch 247
Validation auc = 0.673005
Epoch 248
Validation auc = 0.682864
Epoch 249
Loss = 3.8466e-01, PNorm = 51.4317, GNorm = 15.4239, lr_0 = 6.6056e-04
Validation auc = 0.653521
Epoch 250
Validation auc = 0.620188
Epoch 251
Validation auc = 0.619718
Epoch 252
Validation auc = 0.619249
Epoch 253
Validation auc = 0.618779
Epoch 254
Loss = 3.1852e-01, PNorm = 51.4394, GNorm = 17.2241, lr_0 = 6.5491e-04
Validation auc = 0.603756
Epoch 255
Validation auc = 0.591080
Epoch 256
Validation auc = 0.591549
Epoch 257
Validation auc = 0.611268
Epoch 258
Validation auc = 0.637559
Epoch 259
Loss = 2.3983e-01, PNorm = 51.4468, GNorm = 11.5041, lr_0 = 6.4931e-04
Validation auc = 0.662441
Epoch 260
Validation auc = 0.685446
Epoch 261
Validation auc = 0.695305
Epoch 262
Validation auc = 0.704225
Epoch 263
Validation auc = 0.707981
Epoch 264
Loss = 5.5937e-01, PNorm = 51.4542, GNorm = 67.7838, lr_0 = 6.4376e-04
Validation auc = 0.705164
Epoch 265
Validation auc = 0.692019
Epoch 266
Validation auc = 0.675117
Epoch 267
Validation auc = 0.681221
Epoch 268
Validation auc = 0.690610
Epoch 269
Loss = 3.5894e-01, PNorm = 51.4620, GNorm = 30.5509, lr_0 = 6.3826e-04
Validation auc = 0.707512
Epoch 270
Validation auc = 0.715023
Epoch 271
Validation auc = 0.714085
Epoch 272
Validation auc = 0.710798
Epoch 273
Validation auc = 0.695305
Epoch 274
Loss = 1.6910e-01, PNorm = 51.4698, GNorm = 14.2708, lr_0 = 6.3280e-04
Validation auc = 0.687324
Epoch 275
Validation auc = 0.677934
Epoch 276
Validation auc = 0.683099
Epoch 277
Validation auc = 0.693897
Epoch 278
Validation auc = 0.691549
Epoch 279
Loss = 4.2703e-01, PNorm = 51.4774, GNorm = 49.6403, lr_0 = 6.2739e-04
Validation auc = 0.684272
Epoch 280
Validation auc = 0.681690
Epoch 281
Validation auc = 0.677934
Epoch 282
Validation auc = 0.678873
Epoch 283
Validation auc = 0.686385
Epoch 284
Loss = 6.3374e-01, PNorm = 51.4846, GNorm = 46.0511, lr_0 = 6.2203e-04
Validation auc = 0.696714
Epoch 285
Validation auc = 0.707512
Epoch 286
Validation auc = 0.713146
Epoch 287
Validation auc = 0.713615
Epoch 288
Validation auc = 0.711268
Epoch 289
Loss = 2.9825e-01, PNorm = 51.4913, GNorm = 14.3285, lr_0 = 6.1671e-04
Validation auc = 0.711737
Epoch 290
Validation auc = 0.695305
Epoch 291
Validation auc = 0.697653
Epoch 292
Validation auc = 0.707512
Epoch 293
Validation auc = 0.717840
Epoch 294
Loss = 4.7401e-01, PNorm = 51.4974, GNorm = 36.9898, lr_0 = 6.1144e-04
Validation auc = 0.718779
Epoch 295
Validation auc = 0.717840
Epoch 296
Validation auc = 0.716432
Epoch 297
Validation auc = 0.715023
Epoch 298
Validation auc = 0.703756
Epoch 299
Loss = 2.7242e-01, PNorm = 51.5039, GNorm = 28.2978, lr_0 = 6.0621e-04
Validation auc = 0.694836
Model 0 best validation auc = 0.773239 on epoch 39
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.664058
Ensemble test auc = 0.664058
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Validation auc = 0.552582
Epoch 1
Warning: Found a task with predictions all 0s or all 1s
Validation auc = nan
Epoch 2
Validation auc = 0.631690
Epoch 3
Validation auc = 0.593427
Epoch 4
Loss = 1.7690e+02, PNorm = 50.2743, GNorm = 1830.1156, lr_0 = 6.5000e-04
Validation auc = 0.690845
Epoch 5
Validation auc = 0.621362
Epoch 6
Validation auc = 0.744601
Epoch 7
Validation auc = 0.740141
Epoch 8
Validation auc = 0.686150
Epoch 9
Loss = 6.0591e+01, PNorm = 50.4243, GNorm = 1100.6835, lr_0 = 9.9743e-04
Validation auc = 0.580282
Epoch 10
Validation auc = 0.748357
Epoch 11
Validation auc = 0.736150
Epoch 12
Validation auc = 0.610798
Epoch 13
Validation auc = 0.693662
Epoch 14
Loss = 2.1502e+01, PNorm = 50.5778, GNorm = 553.1222, lr_0 = 9.8890e-04
Validation auc = 0.688028
Epoch 15
Validation auc = 0.705164
Epoch 16
Validation auc = 0.585446
Epoch 17
Validation auc = 0.642723
Epoch 18
Validation auc = 0.629343
Epoch 19
Loss = 1.2873e+01, PNorm = 50.6824, GNorm = 739.3738, lr_0 = 9.8045e-04
Validation auc = 0.694366
Epoch 20
Validation auc = 0.676526
Epoch 21
Validation auc = 0.705869
Epoch 22
Validation auc = 0.662441
Epoch 23
Validation auc = 0.707042
Epoch 24
Loss = 1.3181e+01, PNorm = 50.7506, GNorm = 582.4031, lr_0 = 9.7207e-04
Validation auc = 0.688263
Epoch 25
Validation auc = 0.594836
Epoch 26
Validation auc = 0.640141
Epoch 27
Validation auc = 0.644836
Epoch 28
Validation auc = 0.618075
Epoch 29
Loss = 9.3497e+00, PNorm = 50.7976, GNorm = 399.3704, lr_0 = 9.6376e-04
Validation auc = 0.631221
Epoch 30
Validation auc = 0.641080
Epoch 31
Validation auc = 0.608216
Epoch 32
Validation auc = 0.663146
Epoch 33
Validation auc = 0.660563
Epoch 34
Loss = 4.8645e+00, PNorm = 50.8344, GNorm = 112.9240, lr_0 = 9.5552e-04
Validation auc = 0.565493
Epoch 35
Validation auc = 0.676761
Epoch 36
Validation auc = 0.697653
Epoch 37
Validation auc = 0.607512
Epoch 38
Validation auc = 0.601643
Epoch 39
Loss = 1.0656e+01, PNorm = 50.8673, GNorm = 671.3518, lr_0 = 9.4735e-04
Validation auc = 0.719718
Epoch 40
Validation auc = 0.708216
Epoch 41
Validation auc = 0.658216
Epoch 42
Validation auc = 0.591549
Epoch 43
Validation auc = 0.682864
Epoch 44
Loss = 9.2039e+00, PNorm = 50.8978, GNorm = 161.3443, lr_0 = 9.3925e-04
Validation auc = 0.720188
Epoch 45
Validation auc = 0.711972
Epoch 46
Validation auc = 0.612207
Epoch 47
Validation auc = 0.591080
Epoch 48
Validation auc = 0.650235
Epoch 49
Loss = 4.3908e+00, PNorm = 50.9272, GNorm = 191.7219, lr_0 = 9.3122e-04
Validation auc = 0.664554
Epoch 50
Validation auc = 0.602113
Epoch 51
Validation auc = 0.559390
Epoch 52
Validation auc = 0.600939
Epoch 53
Validation auc = 0.659155
Epoch 54
Loss = 4.0063e+00, PNorm = 50.9521, GNorm = 178.1034, lr_0 = 9.2326e-04
Validation auc = 0.687559
Epoch 55
Validation auc = 0.683099
Epoch 56
Validation auc = 0.665258
Epoch 57
Validation auc = 0.656573
Epoch 58
Validation auc = 0.682629
Epoch 59
Loss = 3.8647e+00, PNorm = 50.9733, GNorm = 213.0870, lr_0 = 9.1537e-04
Validation auc = 0.646948
Epoch 60
Validation auc = 0.637559
Epoch 61
Validation auc = 0.690610
Epoch 62
Validation auc = 0.699061
Epoch 63
Validation auc = 0.687324
Epoch 64
Loss = 3.5644e+00, PNorm = 50.9934, GNorm = 129.9636, lr_0 = 9.0754e-04
Validation auc = 0.686620
Epoch 65
Validation auc = 0.683099
Epoch 66
Validation auc = 0.690376
Epoch 67
Validation auc = 0.693897
Epoch 68
Validation auc = 0.662911
Epoch 69
Loss = 3.4802e+00, PNorm = 51.0148, GNorm = 182.7429, lr_0 = 8.9978e-04
Validation auc = 0.646948
Epoch 70
Validation auc = 0.648826
Epoch 71
Validation auc = 0.689437
Epoch 72
Validation auc = 0.714789
Epoch 73
Validation auc = 0.727934
Epoch 74
Loss = 1.6835e+00, PNorm = 51.0347, GNorm = 112.1199, lr_0 = 8.9209e-04
Validation auc = 0.673944
Epoch 75
Validation auc = 0.633803
Epoch 76
Validation auc = 0.658920
Epoch 77
Validation auc = 0.591549
Epoch 78
Validation auc = 0.652817
Epoch 79
Loss = 1.8194e+00, PNorm = 51.0536, GNorm = 127.5392, lr_0 = 8.8447e-04
Validation auc = 0.646479
Epoch 80
Validation auc = 0.679577
Epoch 81
Validation auc = 0.688498
Epoch 82
Validation auc = 0.699296
Epoch 83
Validation auc = 0.684272
Epoch 84
Loss = 2.5542e+00, PNorm = 51.0726, GNorm = 179.9192, lr_0 = 8.7691e-04
Validation auc = 0.674648
Epoch 85
Validation auc = 0.657981
Epoch 86
Validation auc = 0.666901
Epoch 87
Validation auc = 0.650704
Epoch 88
Validation auc = 0.636385
Epoch 89
Loss = 2.7576e+00, PNorm = 51.0921, GNorm = 128.2008, lr_0 = 8.6941e-04
Validation auc = 0.580047
Epoch 90
Validation auc = 0.535915
Epoch 91
Validation auc = 0.574648
Epoch 92
Validation auc = 0.640845
Epoch 93
Validation auc = 0.665258
Epoch 94
Loss = 8.1599e-01, PNorm = 51.1115, GNorm = 91.9581, lr_0 = 8.6198e-04
Validation auc = 0.672066
Epoch 95
Validation auc = 0.679108
Epoch 96
Validation auc = 0.659155
Epoch 97
Validation auc = 0.622770
Epoch 98
Validation auc = 0.606573
Epoch 99
Loss = 1.5861e+00, PNorm = 51.1286, GNorm = 70.0546, lr_0 = 8.5461e-04
Validation auc = 0.617371
Epoch 100
Validation auc = 0.642723
Epoch 101
Validation auc = 0.609390
Epoch 102
Validation auc = 0.628873
Epoch 103
Validation auc = 0.631690
Epoch 104
Loss = 1.3581e+00, PNorm = 51.1448, GNorm = 99.0760, lr_0 = 8.4730e-04
Validation auc = 0.636620
Epoch 105
Validation auc = 0.663615
Epoch 106
Validation auc = 0.650235
Epoch 107
Validation auc = 0.629343
Epoch 108
Validation auc = 0.684038
Epoch 109
Loss = 1.5576e+00, PNorm = 51.1605, GNorm = 74.1152, lr_0 = 8.4006e-04
Validation auc = 0.695070
Epoch 110
Validation auc = 0.696948
Epoch 111
Validation auc = 0.677934
Epoch 112
Validation auc = 0.655399
Epoch 113
Validation auc = 0.656808
Epoch 114
Loss = 1.4813e+00, PNorm = 51.1764, GNorm = 58.6796, lr_0 = 8.3288e-04
Validation auc = 0.686385
Epoch 115
Validation auc = 0.679343
Epoch 116
Validation auc = 0.664319
Epoch 117
Validation auc = 0.666901
Epoch 118
Validation auc = 0.670657
Epoch 119
Loss = 9.5663e-01, PNorm = 51.1899, GNorm = 119.7556, lr_0 = 8.2576e-04
Validation auc = 0.668310
Epoch 120
Validation auc = 0.637793
Epoch 121
Validation auc = 0.630986
Epoch 122
Validation auc = 0.630751
Epoch 123
Validation auc = 0.665962
Epoch 124
Loss = 9.2254e-01, PNorm = 51.2020, GNorm = 68.3247, lr_0 = 8.1870e-04
Validation auc = 0.664085
Epoch 125
Validation auc = 0.668310
Epoch 126
Validation auc = 0.663380
Epoch 127
Validation auc = 0.656808
Epoch 128
Validation auc = 0.671362
Epoch 129
Loss = 1.3301e+00, PNorm = 51.2132, GNorm = 75.0075, lr_0 = 8.1170e-04
Validation auc = 0.675117
Epoch 130
Validation auc = 0.672770
Epoch 131
Validation auc = 0.658451
Epoch 132
Validation auc = 0.647183
Epoch 133
Validation auc = 0.621831
Epoch 134
Loss = 5.8454e-01, PNorm = 51.2221, GNorm = 45.4451, lr_0 = 8.0476e-04
Validation auc = 0.605869
Epoch 135
Validation auc = 0.592723
Epoch 136
Validation auc = 0.651878
Epoch 137
Validation auc = 0.645305
Epoch 138
Validation auc = 0.645775
Epoch 139
Loss = 1.1642e+00, PNorm = 51.2333, GNorm = 50.6073, lr_0 = 7.9788e-04
Validation auc = 0.664319
Epoch 140
Validation auc = 0.698357
Epoch 141
Validation auc = 0.708685
Epoch 142
Validation auc = 0.680516
Epoch 143
Validation auc = 0.633568
Epoch 144
Loss = 1.2279e+00, PNorm = 51.2469, GNorm = 74.7714, lr_0 = 7.9106e-04
Validation auc = 0.597653
Epoch 145
Validation auc = 0.577230
Epoch 146
Validation auc = 0.568545
Epoch 147
Validation auc = 0.576526
Epoch 148
Validation auc = 0.603756
Epoch 149
Loss = 7.2963e-01, PNorm = 51.2579, GNorm = 73.7704, lr_0 = 7.8430e-04
Validation auc = 0.623005
Epoch 150
Validation auc = 0.616901
Epoch 151
Validation auc = 0.634742
Epoch 152
Validation auc = 0.674178
Epoch 153
Validation auc = 0.672300
Epoch 154
Loss = 5.0797e-01, PNorm = 51.2673, GNorm = 46.0119, lr_0 = 7.7759e-04
Validation auc = 0.692958
Epoch 155
Validation auc = 0.700469
Epoch 156
Validation auc = 0.692488
Epoch 157
Validation auc = 0.682629
Epoch 158
Validation auc = 0.669953
Epoch 159
Loss = 1.1547e+00, PNorm = 51.2779, GNorm = 59.6532, lr_0 = 7.7095e-04
Validation auc = 0.660094
Epoch 160
Validation auc = 0.658685
Epoch 161
Validation auc = 0.659155
Epoch 162
Validation auc = 0.642254
Epoch 163
Validation auc = 0.653052
Epoch 164
Loss = 1.1505e+00, PNorm = 51.2889, GNorm = 102.8484, lr_0 = 7.6436e-04
Validation auc = 0.642254
Epoch 165
Validation auc = 0.649531
Epoch 166
Validation auc = 0.646714
Epoch 167
Validation auc = 0.623944
Epoch 168
Validation auc = 0.611268
Epoch 169
Loss = 3.9442e-01, PNorm = 51.2997, GNorm = 77.5593, lr_0 = 7.5782e-04
Validation auc = 0.594836
Epoch 170
Validation auc = 0.598592
Epoch 171
Validation auc = 0.608920
Epoch 172
Validation auc = 0.619249
Epoch 173
Validation auc = 0.635915
Epoch 174
Loss = 8.5542e-01, PNorm = 51.3093, GNorm = 71.7572, lr_0 = 7.5134e-04
Validation auc = 0.643427
Epoch 175
Validation auc = 0.663146
Epoch 176
Validation auc = 0.694836
Epoch 177
Validation auc = 0.713146
Epoch 178
Validation auc = 0.708920
Epoch 179
Loss = 4.0147e-01, PNorm = 51.3176, GNorm = 34.6556, lr_0 = 7.4492e-04
Validation auc = 0.689202
Epoch 180
Validation auc = 0.660094
Epoch 181
Validation auc = 0.647418
Epoch 182
Validation auc = 0.636150
Epoch 183
Validation auc = 0.638967
Epoch 184
Loss = 3.6163e-01, PNorm = 51.3250, GNorm = 28.4476, lr_0 = 7.3855e-04
Validation auc = 0.658216
Epoch 185
Validation auc = 0.666667
Epoch 186
Validation auc = 0.665728
Epoch 187
Validation auc = 0.670423
Epoch 188
Validation auc = 0.677934
Epoch 189
Loss = 2.8811e-01, PNorm = 51.3311, GNorm = 32.9973, lr_0 = 7.3224e-04
Validation auc = 0.681690
Epoch 190
Validation auc = 0.676056
Epoch 191
Validation auc = 0.674648
Epoch 192
Validation auc = 0.675587
Epoch 193
Validation auc = 0.674648
Epoch 194
Loss = 5.9067e-01, PNorm = 51.3379, GNorm = 57.1242, lr_0 = 7.2598e-04
Validation auc = 0.676995
Epoch 195
Validation auc = 0.681221
Epoch 196
Validation auc = 0.675117
Epoch 197
Validation auc = 0.666197
Epoch 198
Validation auc = 0.671831
Epoch 199
Loss = 5.2672e-01, PNorm = 51.3457, GNorm = 19.1091, lr_0 = 7.1977e-04
Validation auc = 0.658685
Epoch 200
Validation auc = 0.646009
Epoch 201
Validation auc = 0.631455
Epoch 202
Validation auc = 0.621596
Epoch 203
Validation auc = 0.623474
Epoch 204
Loss = 3.1069e-01, PNorm = 51.3535, GNorm = 20.9845, lr_0 = 7.1362e-04
Validation auc = 0.645070
Epoch 205
Validation auc = 0.659624
Epoch 206
Validation auc = 0.673239
Epoch 207
Validation auc = 0.673709
Epoch 208
Validation auc = 0.672770
Epoch 209
Loss = 2.5359e-01, PNorm = 51.3608, GNorm = 10.9397, lr_0 = 7.0752e-04
Validation auc = 0.674648
Epoch 210
Validation auc = 0.686854
Epoch 211
Validation auc = 0.695305
Epoch 212
Validation auc = 0.687324
Epoch 213
Validation auc = 0.675587
Epoch 214
Loss = 2.7918e-01, PNorm = 51.3672, GNorm = 44.2169, lr_0 = 7.0147e-04
Validation auc = 0.649765
Epoch 215
Validation auc = 0.659624
Epoch 216
Validation auc = 0.670423
Epoch 217
Validation auc = 0.677934
Epoch 218
Validation auc = 0.673709
Epoch 219
Loss = 4.2435e-01, PNorm = 51.3744, GNorm = 28.1784, lr_0 = 6.9548e-04
Validation auc = 0.675587
Epoch 220
Validation auc = 0.686385
Epoch 221
Validation auc = 0.686854
Epoch 222
Validation auc = 0.683099
Epoch 223
Validation auc = 0.670892
Epoch 224
Loss = 5.0017e-01, PNorm = 51.3817, GNorm = 31.4024, lr_0 = 6.8953e-04
Validation auc = 0.665728
Epoch 225
Validation auc = 0.671362
Epoch 226
Validation auc = 0.663380
Epoch 227
Validation auc = 0.643662
Epoch 228
Validation auc = 0.632394
Epoch 229
Loss = 8.0608e-01, PNorm = 51.3883, GNorm = 39.0400, lr_0 = 6.8364e-04
Validation auc = 0.661972
Epoch 230
Validation auc = 0.682160
Epoch 231
Validation auc = 0.699531
Epoch 232
Validation auc = 0.710329
Epoch 233
Validation auc = 0.732864
Epoch 234
Loss = 6.4691e-01, PNorm = 51.3947, GNorm = 38.1398, lr_0 = 6.7779e-04
Validation auc = 0.729108
Epoch 235
Validation auc = 0.726761
Epoch 236
Validation auc = 0.715962
Epoch 237
Validation auc = 0.710798
Epoch 238
Validation auc = 0.708451
Epoch 239
Loss = 1.6450e-01, PNorm = 51.4009, GNorm = 21.9535, lr_0 = 6.7200e-04
Validation auc = 0.698592
Epoch 240
Validation auc = 0.642723
Epoch 241
Validation auc = 0.579343
Epoch 242
Validation auc = 0.579343
Epoch 243
Validation auc = 0.620188
Epoch 244
Loss = 3.8347e-01, PNorm = 51.4074, GNorm = 23.2645, lr_0 = 6.6625e-04
Validation auc = 0.658216
Epoch 245
Validation auc = 0.692488
Epoch 246
Validation auc = 0.706103
Epoch 247
Validation auc = 0.708920
Epoch 248
Validation auc = 0.706573
Epoch 249
Loss = 3.7941e-01, PNorm = 51.4139, GNorm = 25.2575, lr_0 = 6.6056e-04
Validation auc = 0.706103
Epoch 250
Validation auc = 0.702817
Epoch 251
Validation auc = 0.690141
Epoch 252
Validation auc = 0.691080
Epoch 253
Validation auc = 0.667606
Epoch 254
Loss = 3.3610e-01, PNorm = 51.4210, GNorm = 27.7123, lr_0 = 6.5491e-04
Validation auc = 0.654460
Epoch 255
Validation auc = 0.647887
Epoch 256
Validation auc = 0.653521
Epoch 257
Validation auc = 0.669484
Epoch 258
Validation auc = 0.677934
Epoch 259
Loss = 3.5184e-01, PNorm = 51.4293, GNorm = 30.0255, lr_0 = 6.4931e-04
Validation auc = 0.690141
Epoch 260
Validation auc = 0.684977
Epoch 261
Validation auc = 0.681690
Epoch 262
Validation auc = 0.672300
Epoch 263
Validation auc = 0.650704
Epoch 264
Loss = 2.8555e-01, PNorm = 51.4371, GNorm = 13.0809, lr_0 = 6.4376e-04
Validation auc = 0.630516
Epoch 265
Validation auc = 0.630986
Epoch 266
Validation auc = 0.636620
Epoch 267
Validation auc = 0.635211
Epoch 268
Validation auc = 0.619718
Epoch 269
Loss = 4.0196e-01, PNorm = 51.4457, GNorm = 61.7927, lr_0 = 6.3826e-04
Validation auc = 0.628638
Epoch 270
Validation auc = 0.651174
Epoch 271
Validation auc = 0.650704
Epoch 272
Validation auc = 0.646009
Epoch 273
Validation auc = 0.641784
Epoch 274
Loss = 1.5261e-01, PNorm = 51.4530, GNorm = 12.1847, lr_0 = 6.3280e-04
Validation auc = 0.640376
Epoch 275
Validation auc = 0.650235
Epoch 276
Validation auc = 0.665258
Epoch 277
Validation auc = 0.676526
Epoch 278
Validation auc = 0.687324
Epoch 279
Loss = 2.6824e-01, PNorm = 51.4610, GNorm = 34.3186, lr_0 = 6.2739e-04
Validation auc = 0.685915
Epoch 280
Validation auc = 0.689671
Epoch 281
Validation auc = 0.694366
Epoch 282
Validation auc = 0.696244
Epoch 283
Validation auc = 0.687793
Epoch 284
Loss = 1.0206e+00, PNorm = 51.4686, GNorm = 89.2761, lr_0 = 6.2203e-04
Validation auc = 0.690141
Epoch 285
Validation auc = 0.705164
Epoch 286
Validation auc = 0.710798
Epoch 287
Validation auc = 0.709390
Epoch 288
Validation auc = 0.713615
Epoch 289
Loss = 2.7228e-01, PNorm = 51.4766, GNorm = 35.0826, lr_0 = 6.1671e-04
Validation auc = 0.724883
Epoch 290
Validation auc = 0.723474
Epoch 291
Validation auc = 0.674178
Epoch 292
Validation auc = 0.653521
Epoch 293
Validation auc = 0.637089
Epoch 294
Loss = 2.4590e-01, PNorm = 51.4843, GNorm = 10.3629, lr_0 = 6.1144e-04
Validation auc = 0.643662
Epoch 295
Validation auc = 0.648357
Epoch 296
Validation auc = 0.653052
Epoch 297
Validation auc = 0.657277
Epoch 298
Validation auc = 0.659155
Epoch 299
Loss = 1.8109e-01, PNorm = 51.4918, GNorm = 12.8221, lr_0 = 6.0621e-04
Validation auc = 0.662441
Model 0 best validation auc = 0.748357 on epoch 10
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.729681
Ensemble test auc = 0.729681
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Validation auc = 0.546948
Epoch 1
Validation auc = 0.592488
Epoch 2
Validation auc = 0.496244
Epoch 3
Validation auc = 0.558920
Epoch 4
Loss = 1.6835e+02, PNorm = 50.2765, GNorm = 1006.6960, lr_0 = 6.5000e-04
Validation auc = 0.683803
Epoch 5
Validation auc = 0.593427
Epoch 6
Validation auc = 0.650235
Epoch 7
Validation auc = 0.583568
Epoch 8
Validation auc = 0.621596
Epoch 9
Loss = 3.7306e+01, PNorm = 50.4300, GNorm = 508.8151, lr_0 = 9.9743e-04
Validation auc = 0.727700
Epoch 10
Validation auc = 0.741315
Epoch 11
Validation auc = 0.691784
Epoch 12
Validation auc = 0.658685
Epoch 13
Validation auc = 0.668310
Epoch 14
Loss = 3.4226e+01, PNorm = 50.5877, GNorm = 1386.2081, lr_0 = 9.8890e-04
Validation auc = 0.655399
Epoch 15
Validation auc = 0.644366
Epoch 16
Validation auc = 0.718310
Epoch 17
Validation auc = 0.745540
Epoch 18
Validation auc = 0.739202
Epoch 19
Loss = 1.0515e+01, PNorm = 50.6839, GNorm = 414.2027, lr_0 = 9.8045e-04
Validation auc = 0.717136
Epoch 20
Validation auc = 0.717606
Epoch 21
Validation auc = 0.713380
Epoch 22
Validation auc = 0.711502
Epoch 23
Validation auc = 0.704460
Epoch 24
Loss = 8.6121e+00, PNorm = 50.7405, GNorm = 250.1777, lr_0 = 9.7207e-04
Validation auc = 0.719484
Epoch 25
Validation auc = 0.685915
Epoch 26
Validation auc = 0.678404
Epoch 27
Validation auc = 0.722770
Epoch 28
Validation auc = 0.716197
Epoch 29
Loss = 1.0574e+01, PNorm = 50.7797, GNorm = 189.6997, lr_0 = 9.6376e-04
Validation auc = 0.692019
Epoch 30
Validation auc = 0.610329
Epoch 31
Validation auc = 0.718075
Epoch 32
Validation auc = 0.652582
Epoch 33
Validation auc = 0.702113
Epoch 34
Loss = 6.8053e+00, PNorm = 50.8095, GNorm = 524.3920, lr_0 = 9.5552e-04
Validation auc = 0.660094
Epoch 35
Validation auc = 0.694131
Epoch 36
Validation auc = 0.719249
Epoch 37
Validation auc = 0.696244
Epoch 38
Validation auc = 0.649061
Epoch 39
Loss = 4.9278e+00, PNorm = 50.8339, GNorm = 143.5962, lr_0 = 9.4735e-04
Validation auc = 0.622770
Epoch 40
Validation auc = 0.650235
Epoch 41
Validation auc = 0.627934
Epoch 42
Validation auc = 0.609390
Epoch 43
Validation auc = 0.652582
Epoch 44
Loss = 2.8256e+00, PNorm = 50.8537, GNorm = 121.0452, lr_0 = 9.3925e-04
Validation auc = 0.706103
Epoch 45
Validation auc = 0.728873
Epoch 46
Validation auc = 0.678404
Epoch 47
Validation auc = 0.660798
Epoch 48
Validation auc = 0.556338
Epoch 49
Loss = 3.6536e+00, PNorm = 50.8732, GNorm = 122.1975, lr_0 = 9.3122e-04
Validation auc = 0.570423
Epoch 50
Validation auc = 0.655634
Epoch 51
Validation auc = 0.706573
Epoch 52
Validation auc = 0.732864
Epoch 53
Validation auc = 0.689437
Epoch 54
Loss = 2.6274e+00, PNorm = 50.8924, GNorm = 232.6732, lr_0 = 9.2326e-04
Validation auc = 0.635211
Epoch 55
Validation auc = 0.616432
Epoch 56
Validation auc = 0.550704
Epoch 57
Validation auc = 0.530047
Epoch 58
Validation auc = 0.595305
Epoch 59
Loss = 7.9319e+00, PNorm = 50.9159, GNorm = 300.2712, lr_0 = 9.1537e-04
Validation auc = 0.621127
Epoch 60
Validation auc = 0.607746
Epoch 61
Validation auc = 0.656338
Epoch 62
Validation auc = 0.623709
Epoch 63
Validation auc = 0.651878
Epoch 64
Loss = 1.8393e+00, PNorm = 50.9396, GNorm = 87.3328, lr_0 = 9.0754e-04
Validation auc = 0.629812
Epoch 65
Validation auc = 0.641784
Epoch 66
Validation auc = 0.688732
Epoch 67
Validation auc = 0.677934
Epoch 68
Validation auc = 0.680282
Epoch 69
Loss = 2.3234e+00, PNorm = 50.9597, GNorm = 86.3767, lr_0 = 8.9978e-04
Validation auc = 0.702582
Epoch 70
Validation auc = 0.667606
Epoch 71
Validation auc = 0.605634
Epoch 72
Validation auc = 0.552582
Epoch 73
Validation auc = 0.535681
Epoch 74
Loss = 1.9873e+00, PNorm = 50.9770, GNorm = 149.0988, lr_0 = 8.9209e-04
Validation auc = 0.549765
Epoch 75
Validation auc = 0.659624
Epoch 76
Validation auc = 0.689671
Epoch 77
Validation auc = 0.680282
Epoch 78
Validation auc = 0.658451
Epoch 79
Loss = 3.3518e+00, PNorm = 50.9925, GNorm = 70.5153, lr_0 = 8.8447e-04
Validation auc = 0.649765
Epoch 80
Validation auc = 0.613146
Epoch 81
Validation auc = 0.587089
Epoch 82
Validation auc = 0.618779
Epoch 83
Validation auc = 0.623239
Epoch 84
Loss = 2.2232e+00, PNorm = 51.0096, GNorm = 75.5295, lr_0 = 8.7691e-04
Validation auc = 0.664085
Epoch 85
Validation auc = 0.703756
Epoch 86
Validation auc = 0.703756
Epoch 87
Validation auc = 0.662676
Epoch 88
Validation auc = 0.603756
Epoch 89
Loss = 2.4393e+00, PNorm = 51.0261, GNorm = 82.9043, lr_0 = 8.6941e-04
Validation auc = 0.641784
Epoch 90
Validation auc = 0.693897
Epoch 91
Validation auc = 0.643192
Epoch 92
Validation auc = 0.628638
Epoch 93
Validation auc = 0.655634
Epoch 94
Loss = 1.2453e+00, PNorm = 51.0406, GNorm = 45.6267, lr_0 = 8.6198e-04
Validation auc = 0.594601
Epoch 95
Validation auc = 0.579108
Epoch 96
Validation auc = 0.629577
Epoch 97
Validation auc = 0.633333
Epoch 98
Validation auc = 0.652113
Epoch 99
Loss = 1.6523e+00, PNorm = 51.0543, GNorm = 89.8297, lr_0 = 8.5461e-04
Validation auc = 0.675587
Epoch 100
Validation auc = 0.677465
Epoch 101
Validation auc = 0.670892
Epoch 102
Validation auc = 0.635211
Epoch 103
Validation auc = 0.599296
Epoch 104
Loss = 1.9639e+00, PNorm = 51.0670, GNorm = 110.1969, lr_0 = 8.4730e-04
Validation auc = 0.615962
Epoch 105
Validation auc = 0.642254
Epoch 106
Validation auc = 0.654695
Epoch 107
Validation auc = 0.647887
Epoch 108
Validation auc = 0.653521
Epoch 109
Loss = 9.8646e-01, PNorm = 51.0792, GNorm = 48.6607, lr_0 = 8.4006e-04
Validation auc = 0.641315
Epoch 110
Validation auc = 0.616432
Epoch 111
Validation auc = 0.602347
Epoch 112
Validation auc = 0.522066
Epoch 113
Validation auc = 0.483099
Epoch 114
Loss = 6.9622e-01, PNorm = 51.0908, GNorm = 54.0691, lr_0 = 8.3288e-04
Validation auc = 0.550235
Epoch 115
Validation auc = 0.583568
Epoch 116
Validation auc = 0.633099
Epoch 117
Validation auc = 0.637559
Epoch 118
Validation auc = 0.612207
Epoch 119
Loss = 1.2934e+00, PNorm = 51.1015, GNorm = 78.6472, lr_0 = 8.2576e-04
Validation auc = 0.606103
Epoch 120
Validation auc = 0.654930
Epoch 121
Validation auc = 0.610329
Epoch 122
Validation auc = 0.613615
Epoch 123
Validation auc = 0.643192
Epoch 124
Loss = 3.4379e-01, PNorm = 51.1112, GNorm = 11.5461, lr_0 = 8.1870e-04
Validation auc = 0.659155
Epoch 125
Validation auc = 0.661972
Epoch 126
Validation auc = 0.642254
Epoch 127
Validation auc = 0.620657
Epoch 128
Validation auc = 0.652582
Epoch 129
Loss = 5.2697e-01, PNorm = 51.1209, GNorm = 40.9625, lr_0 = 8.1170e-04
Validation auc = 0.652113
Epoch 130
Validation auc = 0.656808
Epoch 131
Validation auc = 0.644131
Epoch 132
Validation auc = 0.611737
Epoch 133
Validation auc = 0.634272
Epoch 134
Loss = 3.7993e-01, PNorm = 51.1311, GNorm = 30.3222, lr_0 = 8.0476e-04
Validation auc = 0.649765
Epoch 135
Validation auc = 0.660563
Epoch 136
Validation auc = 0.660094
Epoch 137
Validation auc = 0.636620
Epoch 138
Validation auc = 0.623944
Epoch 139
Loss = 3.5471e-01, PNorm = 51.1405, GNorm = 26.1002, lr_0 = 7.9788e-04
Validation auc = 0.599531
Epoch 140
Validation auc = 0.565728
Epoch 141
Validation auc = 0.541784
Epoch 142
Validation auc = 0.588732
Epoch 143
Validation auc = 0.655399
Epoch 144
Loss = 1.7139e+00, PNorm = 51.1519, GNorm = 137.9041, lr_0 = 7.9106e-04
Validation auc = 0.690845
Epoch 145
Validation auc = 0.705869
Epoch 146
Validation auc = 0.717136
Epoch 147
Validation auc = 0.712911
Epoch 148
Validation auc = 0.711737
Epoch 149
Loss = 1.1361e+00, PNorm = 51.1630, GNorm = 68.2216, lr_0 = 7.8430e-04
Validation auc = 0.711737
Epoch 150
Validation auc = 0.699531
Epoch 151
Validation auc = 0.673239
Epoch 152
Validation auc = 0.651174
Epoch 153
Validation auc = 0.650704
Epoch 154
Loss = 4.1021e-01, PNorm = 51.1745, GNorm = 32.2489, lr_0 = 7.7759e-04
Validation auc = 0.669014
Epoch 155
Validation auc = 0.695775
Epoch 156
Validation auc = 0.693662
Epoch 157
Validation auc = 0.694131
Epoch 158
Validation auc = 0.706103
Epoch 159
Loss = 6.9765e-01, PNorm = 51.1844, GNorm = 11.2622, lr_0 = 7.7095e-04
Validation auc = 0.727700
Epoch 160
Validation auc = 0.703756
Epoch 161
Validation auc = 0.711268
Epoch 162
Validation auc = 0.702347
Epoch 163
Validation auc = 0.710798
Epoch 164
Loss = 7.4159e-01, PNorm = 51.1952, GNorm = 72.1712, lr_0 = 7.6436e-04
Validation auc = 0.723005
Epoch 165
Validation auc = 0.723005
Epoch 166
Validation auc = 0.723944
Epoch 167
Validation auc = 0.709859
Epoch 168
Validation auc = 0.709859
Epoch 169
Loss = 5.1438e-01, PNorm = 51.2048, GNorm = 83.2711, lr_0 = 7.5782e-04
Validation auc = 0.706573
Epoch 170
Validation auc = 0.698592
Epoch 171
Validation auc = 0.690610
Epoch 172
Validation auc = 0.669014
Epoch 173
Validation auc = 0.652113
Epoch 174
Loss = 7.3942e-01, PNorm = 51.2127, GNorm = 38.1574, lr_0 = 7.5134e-04
Validation auc = 0.640845
Epoch 175
Validation auc = 0.633333
Epoch 176
Validation auc = 0.638028
Epoch 177
Validation auc = 0.648826
Epoch 178
Validation auc = 0.677934
Epoch 179
Loss = 4.6829e-01, PNorm = 51.2201, GNorm = 24.9762, lr_0 = 7.4492e-04
Validation auc = 0.693897
Epoch 180
Validation auc = 0.705634
Epoch 181
Validation auc = 0.719718
Epoch 182
Validation auc = 0.729577
Epoch 183
Validation auc = 0.723944
Epoch 184
Loss = 5.5270e-01, PNorm = 51.2274, GNorm = 31.1672, lr_0 = 7.3855e-04
Validation auc = 0.713615
Epoch 185
Validation auc = 0.707981
Epoch 186
Validation auc = 0.704225
Epoch 187
Validation auc = 0.683568
Epoch 188
Validation auc = 0.664319
Epoch 189
Loss = 3.8930e-01, PNorm = 51.2341, GNorm = 26.2519, lr_0 = 7.3224e-04
Validation auc = 0.666197
Epoch 190
Validation auc = 0.674178
Epoch 191
Validation auc = 0.675587
Epoch 192
Validation auc = 0.681690
Epoch 193
Validation auc = 0.696244
Epoch 194
Loss = 4.9677e-01, PNorm = 51.2412, GNorm = 30.9445, lr_0 = 7.2598e-04
Validation auc = 0.683099
Epoch 195
Validation auc = 0.657746
Epoch 196
Validation auc = 0.650704
Epoch 197
Validation auc = 0.634272
Epoch 198
Validation auc = 0.615023
Epoch 199
Loss = 3.5353e-01, PNorm = 51.2475, GNorm = 26.4203, lr_0 = 7.1977e-04
Validation auc = 0.611737
Epoch 200
Validation auc = 0.610329
Epoch 201
Validation auc = 0.630516
Epoch 202
Validation auc = 0.656338
Epoch 203
Validation auc = 0.670423
Epoch 204
Loss = 3.4022e-01, PNorm = 51.2530, GNorm = 19.4224, lr_0 = 7.1362e-04
Validation auc = 0.680751
Epoch 205
Validation auc = 0.677465
Epoch 206
Validation auc = 0.679343
Epoch 207
Validation auc = 0.685446
Epoch 208
Validation auc = 0.693427
Epoch 209
Loss = 5.9400e-01, PNorm = 51.2590, GNorm = 40.4762, lr_0 = 7.0752e-04
Validation auc = 0.685915
Epoch 210
Validation auc = 0.677465
Epoch 211
Validation auc = 0.671831
Epoch 212
Validation auc = 0.658685
Epoch 213
Validation auc = 0.658216
Epoch 214
Loss = 6.9522e-01, PNorm = 51.2660, GNorm = 29.1939, lr_0 = 7.0147e-04
Validation auc = 0.662441
Epoch 215
Validation auc = 0.668545
Epoch 216
Validation auc = 0.669953
Epoch 217
Validation auc = 0.678873
Epoch 218
Validation auc = 0.676526
Epoch 219
Loss = 3.3854e-01, PNorm = 51.2733, GNorm = 19.8965, lr_0 = 6.9548e-04
Validation auc = 0.670892
Epoch 220
Validation auc = 0.655399
Epoch 221
Validation auc = 0.652113
Epoch 222
Validation auc = 0.647418
Epoch 223
Validation auc = 0.660563
Epoch 224
Loss = 5.0386e-01, PNorm = 51.2806, GNorm = 43.8319, lr_0 = 6.8953e-04
Validation auc = 0.672300
Epoch 225
Validation auc = 0.681690
Epoch 226
Validation auc = 0.687324
Epoch 227
Validation auc = 0.678404
Epoch 228
Validation auc = 0.679812
Epoch 229
Loss = 1.1656e-01, PNorm = 51.2877, GNorm = 38.0208, lr_0 = 6.8364e-04
Validation auc = 0.682629
Epoch 230
Validation auc = 0.678873
Epoch 231
Validation auc = 0.668075
Epoch 232
Validation auc = 0.670892
Epoch 233
Validation auc = 0.673239
Epoch 234
Loss = 3.4341e-01, PNorm = 51.2947, GNorm = 27.2875, lr_0 = 6.7779e-04
Validation auc = 0.671831
Epoch 235
Validation auc = 0.669484
Epoch 236
Validation auc = 0.666667
Epoch 237
Validation auc = 0.663850
Epoch 238
Validation auc = 0.654930
Epoch 239
Loss = 2.0601e-01, PNorm = 51.3008, GNorm = 14.4414, lr_0 = 6.7200e-04
Validation auc = 0.647887
Epoch 240
Validation auc = 0.642254
Epoch 241
Validation auc = 0.643192
Epoch 242
Validation auc = 0.648357
Epoch 243
Validation auc = 0.662441
Epoch 244
Loss = 4.4142e-01, PNorm = 51.3066, GNorm = 35.1220, lr_0 = 6.6625e-04
Validation auc = 0.663380
Epoch 245
Validation auc = 0.656338
Epoch 246
Validation auc = 0.654930
Epoch 247
Validation auc = 0.656808
Epoch 248
Validation auc = 0.655399
Epoch 249
Loss = 3.9337e-01, PNorm = 51.3134, GNorm = 53.0558, lr_0 = 6.6056e-04
Validation auc = 0.662441
Epoch 250
Validation auc = 0.664319
Epoch 251
Validation auc = 0.656338
Epoch 252
Validation auc = 0.649765
Epoch 253
Validation auc = 0.643662
Epoch 254
Loss = 2.5019e-01, PNorm = 51.3207, GNorm = 22.4148, lr_0 = 6.5491e-04
Validation auc = 0.648357
Epoch 255
Validation auc = 0.648826
Epoch 256
Validation auc = 0.656808
Epoch 257
Validation auc = 0.656808
Epoch 258
Validation auc = 0.655399
Epoch 259
Loss = 1.7483e-01, PNorm = 51.3272, GNorm = 9.0667, lr_0 = 6.4931e-04
Validation auc = 0.653991
Epoch 260
Validation auc = 0.658685
Epoch 261
Validation auc = 0.664319
Epoch 262
Validation auc = 0.663380
Epoch 263
Validation auc = 0.667136
Epoch 264
Loss = 2.4993e-01, PNorm = 51.3330, GNorm = 16.2145, lr_0 = 6.4376e-04
Validation auc = 0.676056
Epoch 265
Validation auc = 0.677465
Epoch 266
Validation auc = 0.674648
Epoch 267
Validation auc = 0.661033
Epoch 268
Validation auc = 0.645070
Epoch 269
Loss = 2.9184e-01, PNorm = 51.3384, GNorm = 38.8899, lr_0 = 6.3826e-04
Validation auc = 0.651643
Epoch 270
Validation auc = 0.658216
Epoch 271
Validation auc = 0.660563
Epoch 272
Validation auc = 0.661033
Epoch 273
Validation auc = 0.664789
Epoch 274
Loss = 1.4026e-01, PNorm = 51.3454, GNorm = 9.1466, lr_0 = 6.3280e-04
Validation auc = 0.666667
Epoch 275
Validation auc = 0.657746
Epoch 276
Validation auc = 0.647418
Epoch 277
Validation auc = 0.639437
Epoch 278
Validation auc = 0.634272
Epoch 279
Loss = 1.4632e-01, PNorm = 51.3512, GNorm = 8.9689, lr_0 = 6.2739e-04
Validation auc = 0.626761
Epoch 280
Validation auc = 0.623474
Epoch 281
Validation auc = 0.617840
Epoch 282
Validation auc = 0.623474
Epoch 283
Validation auc = 0.631925
Epoch 284
Loss = 3.6112e-01, PNorm = 51.3568, GNorm = 37.6568, lr_0 = 6.2203e-04
Validation auc = 0.629577
Epoch 285
Validation auc = 0.635211
Epoch 286
Validation auc = 0.638028
Epoch 287
Validation auc = 0.642723
Epoch 288
Validation auc = 0.653052
Epoch 289
Loss = 2.5217e-01, PNorm = 51.3626, GNorm = 14.4931, lr_0 = 6.1671e-04
Validation auc = 0.662911
Epoch 290
Validation auc = 0.670423
Epoch 291
Validation auc = 0.675587
Epoch 292
Validation auc = 0.669953
Epoch 293
Validation auc = 0.657277
Epoch 294
Loss = 2.7426e-01, PNorm = 51.3676, GNorm = 29.1433, lr_0 = 6.1144e-04
Validation auc = 0.649296
Epoch 295
Validation auc = 0.646948
Epoch 296
Validation auc = 0.654460
Epoch 297
Validation auc = 0.661972
Epoch 298
Validation auc = 0.670892
Epoch 299
Loss = 2.4874e-01, PNorm = 51.3732, GNorm = 18.6516, lr_0 = 6.0621e-04
Validation auc = 0.670423
Model 0 best validation auc = 0.745540 on epoch 17
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.723359
Ensemble test auc = 0.723359
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Warning: Found a task with predictions all 0s or all 1s
Validation auc = nan
Epoch 1
Validation auc = 0.671362
Epoch 2
Validation auc = 0.640376
Epoch 3
Validation auc = 0.685446
Epoch 4
Loss = 3.6190e+01, PNorm = 50.2706, GNorm = 644.6348, lr_0 = 6.5000e-04
Validation auc = 0.718779
Epoch 5
Validation auc = 0.666197
Epoch 6
Validation auc = 0.613850
Epoch 7
Validation auc = 0.603991
Epoch 8
Validation auc = 0.714319
Epoch 9
Loss = 2.2410e+01, PNorm = 50.4032, GNorm = 783.6574, lr_0 = 9.9743e-04
Validation auc = 0.683333
Epoch 10
Validation auc = 0.623944
Epoch 11
Validation auc = 0.593662
Epoch 12
Validation auc = 0.641784
Epoch 13
Validation auc = 0.631455
Epoch 14
Loss = 8.7241e+01, PNorm = 50.5602, GNorm = 2826.1360, lr_0 = 9.8890e-04
Validation auc = 0.693897
Epoch 15
Validation auc = 0.671362
Epoch 16
Validation auc = 0.671596
Epoch 17
Validation auc = 0.670892
Epoch 18
Validation auc = 0.642254
Epoch 19
Loss = 3.4298e+01, PNorm = 50.6867, GNorm = 711.4954, lr_0 = 9.8045e-04
Validation auc = 0.626761
Epoch 20
Validation auc = 0.693427
Epoch 21
Validation auc = 0.700235
Epoch 22
Validation auc = 0.615728
Epoch 23
Validation auc = 0.664319
Epoch 24
Loss = 2.3645e+01, PNorm = 50.7694, GNorm = 1088.9171, lr_0 = 9.7207e-04
Validation auc = 0.723709
Epoch 25
Validation auc = 0.735211
Epoch 26
Validation auc = 0.582629
Epoch 27
Validation auc = 0.703521
Epoch 28
Validation auc = 0.747887
Epoch 29
Loss = 1.7834e+01, PNorm = 50.8312, GNorm = 263.2219, lr_0 = 9.6376e-04
Validation auc = 0.652582
Epoch 30
Validation auc = 0.646714
Epoch 31
Validation auc = 0.744366
Epoch 32
Validation auc = 0.690141
Epoch 33
Validation auc = 0.649531
Epoch 34
Loss = 1.2551e+01, PNorm = 50.8812, GNorm = 176.2186, lr_0 = 9.5552e-04
Validation auc = 0.660563
Epoch 35
Validation auc = 0.650235
Epoch 36
Validation auc = 0.662676
Epoch 37
Validation auc = 0.691080
Epoch 38
Validation auc = 0.733099
Epoch 39
Loss = 1.4393e+01, PNorm = 50.9193, GNorm = 599.7513, lr_0 = 9.4735e-04
Validation auc = 0.722535
Epoch 40
Validation auc = 0.667371
Epoch 41
Validation auc = 0.719484
Epoch 42
Validation auc = 0.675587
Epoch 43
Validation auc = 0.713380
Epoch 44
Loss = 6.1963e+00, PNorm = 50.9536, GNorm = 140.6036, lr_0 = 9.3925e-04
Validation auc = 0.685211
Epoch 45
Validation auc = 0.657042
Epoch 46
Validation auc = 0.613615
Epoch 47
Validation auc = 0.662441
Epoch 48
Validation auc = 0.728404
Epoch 49
Loss = 7.0727e+00, PNorm = 50.9841, GNorm = 495.2187, lr_0 = 9.3122e-04
Validation auc = 0.707981
Epoch 50
Validation auc = 0.683803
Epoch 51
Validation auc = 0.724883
Epoch 52
Validation auc = 0.695775
Epoch 53
Validation auc = 0.640141
Epoch 54
Loss = 1.6262e+01, PNorm = 51.0134, GNorm = 578.6672, lr_0 = 9.2326e-04
Validation auc = 0.645305
Epoch 55
Validation auc = 0.697653
Epoch 56
Validation auc = 0.727230
Epoch 57
Validation auc = 0.690845
Epoch 58
Validation auc = 0.658685
Epoch 59
Loss = 4.7653e+00, PNorm = 51.0421, GNorm = 167.9907, lr_0 = 9.1537e-04
Validation auc = 0.688967
Epoch 60
Validation auc = 0.735446
Epoch 61
Validation auc = 0.723239
Epoch 62
Validation auc = 0.682864
Epoch 63
Validation auc = 0.589906
Epoch 64
Loss = 3.1577e+00, PNorm = 51.0664, GNorm = 60.9020, lr_0 = 9.0754e-04
Validation auc = 0.556573
Epoch 65
Validation auc = 0.644131
Epoch 66
Validation auc = 0.646009
Epoch 67
Validation auc = 0.703052
Epoch 68
Validation auc = 0.699531
Epoch 69
Loss = 5.2984e+00, PNorm = 51.0866, GNorm = 219.0516, lr_0 = 8.9978e-04
Validation auc = 0.676761
Epoch 70
Validation auc = 0.668545
Epoch 71
Validation auc = 0.676291
Epoch 72
Validation auc = 0.675822
Epoch 73
Validation auc = 0.658920
Epoch 74
Loss = 3.8784e+00, PNorm = 51.1073, GNorm = 174.7597, lr_0 = 8.9209e-04
Validation auc = 0.646714
Epoch 75
Validation auc = 0.642723
Epoch 76
Validation auc = 0.642958
Epoch 77
Validation auc = 0.620657
Epoch 78
Validation auc = 0.638967
Epoch 79
Loss = 3.0827e+00, PNorm = 51.1292, GNorm = 139.8946, lr_0 = 8.8447e-04
Validation auc = 0.661268
Epoch 80
Validation auc = 0.671127
Epoch 81
Validation auc = 0.675352
Epoch 82
Validation auc = 0.674883
Epoch 83
Validation auc = 0.641080
Epoch 84
Loss = 2.3459e+00, PNorm = 51.1511, GNorm = 237.5390, lr_0 = 8.7691e-04
Validation auc = 0.616432
Epoch 85
Validation auc = 0.629343
Epoch 86
Validation auc = 0.614319
Epoch 87
Validation auc = 0.606103
Epoch 88
Validation auc = 0.604225
Epoch 89
Loss = 2.1257e+00, PNorm = 51.1716, GNorm = 133.4304, lr_0 = 8.6941e-04
Validation auc = 0.641080
Epoch 90
Validation auc = 0.682629
Epoch 91
Validation auc = 0.663380
Epoch 92
Validation auc = 0.673709
Epoch 93
Validation auc = 0.718779
Epoch 94
Loss = 3.8283e+00, PNorm = 51.1944, GNorm = 258.8543, lr_0 = 8.6198e-04
Validation auc = 0.724648
Epoch 95
Validation auc = 0.707042
Epoch 96
Validation auc = 0.668075
Epoch 97
Validation auc = 0.593897
Epoch 98
Validation auc = 0.587793
Epoch 99
Loss = 2.6972e+00, PNorm = 51.2172, GNorm = 92.7961, lr_0 = 8.5461e-04
Validation auc = 0.623005
Epoch 100
Validation auc = 0.636385
Epoch 101
Validation auc = 0.693192
Epoch 102
Validation auc = 0.718310
Epoch 103
Validation auc = 0.697887
Epoch 104
Loss = 3.2002e+00, PNorm = 51.2390, GNorm = 138.8230, lr_0 = 8.4730e-04
Validation auc = 0.659624
Epoch 105
Validation auc = 0.670892
Epoch 106
Validation auc = 0.700939
Epoch 107
Validation auc = 0.623709
Epoch 108
Validation auc = 0.674413
Epoch 109
Loss = 1.0919e+00, PNorm = 51.2584, GNorm = 99.5788, lr_0 = 8.4006e-04
Validation auc = 0.680282
Epoch 110
Validation auc = 0.693427
Epoch 111
Validation auc = 0.711972
Epoch 112
Validation auc = 0.716432
Epoch 113
Validation auc = 0.716432
Epoch 114
Loss = 4.7541e+00, PNorm = 51.2776, GNorm = 452.5016, lr_0 = 8.3288e-04
Validation auc = 0.702817
Epoch 115
Validation auc = 0.693662
Epoch 116
Validation auc = 0.700000
Epoch 117
Validation auc = 0.720892
Epoch 118
Validation auc = 0.732864
Epoch 119
Loss = 2.1244e+00, PNorm = 51.2965, GNorm = 145.4567, lr_0 = 8.2576e-04
Validation auc = 0.730282
Epoch 120
Validation auc = 0.707042
Epoch 121
Validation auc = 0.684742
Epoch 122
Validation auc = 0.690845
Epoch 123
Validation auc = 0.691080
Epoch 124
Loss = 1.7759e+00, PNorm = 51.3139, GNorm = 211.0189, lr_0 = 8.1870e-04
Validation auc = 0.693897
Epoch 125
Validation auc = 0.684742
Epoch 126
Validation auc = 0.692488
Epoch 127
Validation auc = 0.716197
Epoch 128
Validation auc = 0.724648
Epoch 129
Loss = 8.0253e-01, PNorm = 51.3298, GNorm = 88.3762, lr_0 = 8.1170e-04
Validation auc = 0.727465
Epoch 130
Validation auc = 0.710563
Epoch 131
Validation auc = 0.689671
Epoch 132
Validation auc = 0.664319
Epoch 133
Validation auc = 0.648357
Epoch 134
Loss = 8.1526e-01, PNorm = 51.3458, GNorm = 96.2455, lr_0 = 8.0476e-04
Validation auc = 0.657981
Epoch 135
Validation auc = 0.663615
Epoch 136
Validation auc = 0.679812
Epoch 137
Validation auc = 0.703052
Epoch 138
Validation auc = 0.711033
Epoch 139
Loss = 1.0020e+00, PNorm = 51.3594, GNorm = 134.0184, lr_0 = 7.9788e-04
Validation auc = 0.715962
Epoch 140
Validation auc = 0.704695
Epoch 141
Validation auc = 0.706808
Epoch 142
Validation auc = 0.707277
Epoch 143
Validation auc = 0.714319
Epoch 144
Loss = 1.4908e+00, PNorm = 51.3716, GNorm = 33.6424, lr_0 = 7.9106e-04
Validation auc = 0.712676
Epoch 145
Validation auc = 0.713615
Epoch 146
Validation auc = 0.696714
Epoch 147
Validation auc = 0.680047
Epoch 148
Validation auc = 0.682629
Epoch 149
Loss = 1.3585e+00, PNorm = 51.3829, GNorm = 95.2656, lr_0 = 7.8430e-04
Validation auc = 0.696714
Epoch 150
Validation auc = 0.690141
Epoch 151
Validation auc = 0.676056
Epoch 152
Validation auc = 0.655634
Epoch 153
Validation auc = 0.642958
Epoch 154
Loss = 8.4877e-01, PNorm = 51.3934, GNorm = 41.4254, lr_0 = 7.7759e-04
Validation auc = 0.643427
Epoch 155
Validation auc = 0.655634
Epoch 156
Validation auc = 0.658685
Epoch 157
Validation auc = 0.676526
Epoch 158
Validation auc = 0.681455
Epoch 159
Loss = 5.3677e-01, PNorm = 51.4039, GNorm = 83.0205, lr_0 = 7.7095e-04
Validation auc = 0.690141
Epoch 160
Validation auc = 0.702582
Epoch 161
Validation auc = 0.712676
Epoch 162
Validation auc = 0.713615
Epoch 163
Validation auc = 0.714319
Epoch 164
Loss = 1.5145e+00, PNorm = 51.4147, GNorm = 76.9603, lr_0 = 7.6436e-04
Validation auc = 0.705869
Epoch 165
Validation auc = 0.698122
Epoch 166
Validation auc = 0.690610
Epoch 167
Validation auc = 0.671596
Epoch 168
Validation auc = 0.660329
Epoch 169
Loss = 3.2384e-01, PNorm = 51.4243, GNorm = 27.5616, lr_0 = 7.5782e-04
Validation auc = 0.680516
Epoch 170
Validation auc = 0.686385
Epoch 171
Validation auc = 0.686150
Epoch 172
Validation auc = 0.660329
Epoch 173
Validation auc = 0.651408
Epoch 174
Loss = 8.6843e-01, PNorm = 51.4327, GNorm = 115.6335, lr_0 = 7.5134e-04
Validation auc = 0.653521
Epoch 175
Validation auc = 0.669484
Epoch 176
Validation auc = 0.671362
Epoch 177
Validation auc = 0.670892
Epoch 178
Validation auc = 0.661502
Epoch 179
Loss = 6.5394e-01, PNorm = 51.4407, GNorm = 24.2396, lr_0 = 7.4492e-04
Validation auc = 0.650235
Epoch 180
Validation auc = 0.642958
Epoch 181
Validation auc = 0.636620
Epoch 182
Validation auc = 0.638263
Epoch 183
Validation auc = 0.639202
Epoch 184
Loss = 5.2521e-01, PNorm = 51.4498, GNorm = 16.3037, lr_0 = 7.3855e-04
Validation auc = 0.631455
Epoch 185
Validation auc = 0.642723
Epoch 186
Validation auc = 0.652582
Epoch 187
Validation auc = 0.646009
Epoch 188
Validation auc = 0.644601
Epoch 189
Loss = 6.6749e-01, PNorm = 51.4590, GNorm = 61.4884, lr_0 = 7.3224e-04
Validation auc = 0.646009
Epoch 190
Validation auc = 0.651174
Epoch 191
Validation auc = 0.669249
Epoch 192
Validation auc = 0.669718
Epoch 193
Validation auc = 0.653521
Epoch 194
Loss = 6.2717e-01, PNorm = 51.4684, GNorm = 81.9911, lr_0 = 7.2598e-04
Validation auc = 0.655399
Epoch 195
Validation auc = 0.650704
Epoch 196
Validation auc = 0.626761
Epoch 197
Validation auc = 0.630986
Epoch 198
Validation auc = 0.628638
Epoch 199
Loss = 6.4539e-01, PNorm = 51.4776, GNorm = 71.3934, lr_0 = 7.1977e-04
Validation auc = 0.628169
Epoch 200
Validation auc = 0.646479
Epoch 201
Validation auc = 0.667136
Epoch 202
Validation auc = 0.676526
Epoch 203
Validation auc = 0.677465
Epoch 204
Loss = 5.2840e-01, PNorm = 51.4863, GNorm = 65.5855, lr_0 = 7.1362e-04
Validation auc = 0.694836
Epoch 205
Validation auc = 0.700939
Epoch 206
Validation auc = 0.715493
Epoch 207
Validation auc = 0.724413
Epoch 208
Validation auc = 0.726291
Epoch 209
Loss = 4.7845e-01, PNorm = 51.4941, GNorm = 62.9729, lr_0 = 7.0752e-04
Validation auc = 0.726761
Epoch 210
Validation auc = 0.716432
Epoch 211
Validation auc = 0.710329
Epoch 212
Validation auc = 0.714085
Epoch 213
Validation auc = 0.714554
Epoch 214
Loss = 4.2626e-01, PNorm = 51.5029, GNorm = 43.2667, lr_0 = 7.0147e-04
Validation auc = 0.680751
Epoch 215
Validation auc = 0.666667
Epoch 216
Validation auc = 0.631925
Epoch 217
Validation auc = 0.617371
Epoch 218
Validation auc = 0.584977
Epoch 219
Loss = 1.9366e-01, PNorm = 51.5121, GNorm = 70.5317, lr_0 = 6.9548e-04
Validation auc = 0.578404
Epoch 220
Validation auc = 0.595305
Epoch 221
Validation auc = 0.610798
Epoch 222
Validation auc = 0.616901
Epoch 223
Validation auc = 0.645540
Epoch 224
Loss = 5.8358e-01, PNorm = 51.5211, GNorm = 34.1407, lr_0 = 6.8953e-04
Validation auc = 0.657277
Epoch 225
Validation auc = 0.671362
Epoch 226
Validation auc = 0.682160
Epoch 227
Validation auc = 0.686385
Epoch 228
Validation auc = 0.703286
Epoch 229
Loss = 5.1554e-01, PNorm = 51.5293, GNorm = 77.4173, lr_0 = 6.8364e-04
Validation auc = 0.702817
Epoch 230
Validation auc = 0.698122
Epoch 231
Validation auc = 0.698592
Epoch 232
Validation auc = 0.679812
Epoch 233
Validation auc = 0.608451
Epoch 234
Loss = 8.0478e-01, PNorm = 51.5368, GNorm = 105.8882, lr_0 = 6.7779e-04
Validation auc = 0.582160
Epoch 235
Validation auc = 0.660094
Epoch 236
Validation auc = 0.668075
Epoch 237
Validation auc = 0.661972
Epoch 238
Validation auc = 0.661502
Epoch 239
Loss = 1.1506e+00, PNorm = 51.5449, GNorm = 34.9940, lr_0 = 6.7200e-04
Validation auc = 0.653052
Epoch 240
Validation auc = 0.666667
Epoch 241
Validation auc = 0.693662
Epoch 242
Validation auc = 0.698592
Epoch 243
Validation auc = 0.687324
Epoch 244
Loss = 7.2518e-01, PNorm = 51.5527, GNorm = 37.2519, lr_0 = 6.6625e-04
Validation auc = 0.686854
Epoch 245
Validation auc = 0.692488
Epoch 246
Validation auc = 0.701408
Epoch 247
Validation auc = 0.698122
Epoch 248
Validation auc = 0.699531
Epoch 249
Loss = 5.2135e-01, PNorm = 51.5600, GNorm = 45.5652, lr_0 = 6.6056e-04
Validation auc = 0.699531
Epoch 250
Validation auc = 0.700469
Epoch 251
Validation auc = 0.696244
Epoch 252
Validation auc = 0.694366
Epoch 253
Validation auc = 0.689202
Epoch 254
Loss = 2.2512e-01, PNorm = 51.5675, GNorm = 35.8552, lr_0 = 6.5491e-04
Validation auc = 0.681690
Epoch 255
Validation auc = 0.676995
Epoch 256
Validation auc = 0.671831
Epoch 257
Validation auc = 0.668075
Epoch 258
Validation auc = 0.660094
Epoch 259
Loss = 4.6299e-01, PNorm = 51.5745, GNorm = 40.6637, lr_0 = 6.4931e-04
Validation auc = 0.655399
Epoch 260
Validation auc = 0.660094
Epoch 261
Validation auc = 0.659624
Epoch 262
Validation auc = 0.646009
Epoch 263
Validation auc = 0.638967
Epoch 264
Loss = 5.4703e-01, PNorm = 51.5810, GNorm = 39.4946, lr_0 = 6.4376e-04
Validation auc = 0.654930
Epoch 265
Validation auc = 0.672300
Epoch 266
Validation auc = 0.677465
Epoch 267
Validation auc = 0.684038
Epoch 268
Validation auc = 0.714085
Epoch 269
Loss = 1.7837e-01, PNorm = 51.5878, GNorm = 34.9311, lr_0 = 6.3826e-04
Validation auc = 0.721127
Epoch 270
Validation auc = 0.719484
Epoch 271
Validation auc = 0.733333
Epoch 272
Validation auc = 0.734742
Epoch 273
Validation auc = 0.724413
Epoch 274
Loss = 6.0926e-01, PNorm = 51.5941, GNorm = 42.2863, lr_0 = 6.3280e-04
Validation auc = 0.719249
Epoch 275
Validation auc = 0.708920
Epoch 276
Validation auc = 0.700939
Epoch 277
Validation auc = 0.691080
Epoch 278
Validation auc = 0.681690
Epoch 279
Loss = 3.8327e-01, PNorm = 51.6004, GNorm = 42.4656, lr_0 = 6.2739e-04
Validation auc = 0.680282
Epoch 280
Validation auc = 0.663380
Epoch 281
Validation auc = 0.661502
Epoch 282
Validation auc = 0.661502
Epoch 283
Validation auc = 0.657746
Epoch 284
Loss = 5.0321e-01, PNorm = 51.6063, GNorm = 63.4092, lr_0 = 6.2203e-04
Validation auc = 0.654930
Epoch 285
Validation auc = 0.668545
Epoch 286
Validation auc = 0.677934
Epoch 287
Validation auc = 0.683099
Epoch 288
Validation auc = 0.685915
Epoch 289
Loss = 2.6832e-01, PNorm = 51.6120, GNorm = 14.7173, lr_0 = 6.1671e-04
Validation auc = 0.682629
Epoch 290
Validation auc = 0.677934
Epoch 291
Validation auc = 0.670892
Epoch 292
Validation auc = 0.669484
Epoch 293
Validation auc = 0.669953
Epoch 294
Loss = 9.4776e-01, PNorm = 51.6172, GNorm = 47.3035, lr_0 = 6.1144e-04
Validation auc = 0.655399
Epoch 295
Validation auc = 0.644601
Epoch 296
Validation auc = 0.635211
Epoch 297
Validation auc = 0.623474
Epoch 298
Validation auc = 0.623474
Epoch 299
Loss = 4.5633e-01, PNorm = 51.6228, GNorm = 37.6387, lr_0 = 6.0621e-04
Validation auc = 0.638498
Model 0 best validation auc = 0.747887 on epoch 28
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.732390
Ensemble test auc = 0.732390
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Validation auc = 0.672535
Epoch 1
Validation auc = 0.689906
Epoch 2
Validation auc = 0.746244
Epoch 3
Validation auc = 0.687793
Epoch 4
Loss = 4.3143e+01, PNorm = 50.2699, GNorm = 1909.6231, lr_0 = 6.5000e-04
Validation auc = 0.747183
Epoch 5
Validation auc = 0.647418
Epoch 6
Validation auc = 0.641784
Epoch 7
Validation auc = 0.485446
Epoch 8
Validation auc = 0.750000
Epoch 9
Loss = 7.6556e+01, PNorm = 50.4021, GNorm = 998.4177, lr_0 = 9.9743e-04
Validation auc = 0.612207
Epoch 10
Validation auc = 0.676526
Epoch 11
Validation auc = 0.726995
Epoch 12
Validation auc = 0.557746
Epoch 13
Validation auc = 0.647887
Epoch 14
Loss = 3.3590e+01, PNorm = 50.5607, GNorm = 710.7659, lr_0 = 9.8890e-04
Validation auc = 0.718310
Epoch 15
Validation auc = 0.740610
Epoch 16
Validation auc = 0.681690
Epoch 17
Validation auc = 0.729812
Epoch 18
Validation auc = 0.706338
Epoch 19
Loss = 2.3679e+01, PNorm = 50.6760, GNorm = 382.1021, lr_0 = 9.8045e-04
Validation auc = 0.676995
Epoch 20
Validation auc = 0.609390
Epoch 21
Validation auc = 0.656573
Epoch 22
Validation auc = 0.552113
Epoch 23
Validation auc = 0.727700
Epoch 24
Loss = 1.4518e+01, PNorm = 50.7584, GNorm = 795.5797, lr_0 = 9.7207e-04
Validation auc = 0.749296
Epoch 25
Validation auc = 0.670892
Epoch 26
Validation auc = 0.636150
Epoch 27
Validation auc = 0.723474
Epoch 28
Validation auc = 0.765493
Epoch 29
Loss = 1.9865e+01, PNorm = 50.8238, GNorm = 471.5315, lr_0 = 9.6376e-04
Validation auc = 0.723239
Epoch 30
Validation auc = 0.660563
Epoch 31
Validation auc = 0.671596
Epoch 32
Validation auc = 0.710094
Epoch 33
Validation auc = 0.713615
Epoch 34
Loss = 6.6371e+00, PNorm = 50.8694, GNorm = 282.6798, lr_0 = 9.5552e-04
Validation auc = 0.687324
Epoch 35
Validation auc = 0.719249
Epoch 36
Validation auc = 0.719718
Epoch 37
Validation auc = 0.692723
Epoch 38
Validation auc = 0.690376
Epoch 39
Loss = 5.4576e+00, PNorm = 50.9054, GNorm = 225.7088, lr_0 = 9.4735e-04
Validation auc = 0.712207
Epoch 40
Validation auc = 0.626056
Epoch 41
Validation auc = 0.602347
Epoch 42
Validation auc = 0.703991
Epoch 43
Validation auc = 0.677230
Epoch 44
Loss = 4.7671e+00, PNorm = 50.9378, GNorm = 118.0633, lr_0 = 9.3925e-04
Validation auc = 0.647183
Epoch 45
Validation auc = 0.654225
Epoch 46
Validation auc = 0.693897
Epoch 47
Validation auc = 0.740845
Epoch 48
Validation auc = 0.731690
Epoch 49
Loss = 3.2677e+00, PNorm = 50.9686, GNorm = 164.8747, lr_0 = 9.3122e-04
Validation auc = 0.726995
Epoch 50
Validation auc = 0.743897
Epoch 51
Validation auc = 0.714789
Epoch 52
Validation auc = 0.689202
Epoch 53
Validation auc = 0.696244
Epoch 54
Loss = 4.3551e+00, PNorm = 50.9974, GNorm = 198.2005, lr_0 = 9.2326e-04
Validation auc = 0.659155
Epoch 55
Validation auc = 0.658685
Epoch 56
Validation auc = 0.683803
Epoch 57
Validation auc = 0.668075
Epoch 58
Validation auc = 0.678873
Epoch 59
Loss = 4.3723e+00, PNorm = 51.0232, GNorm = 338.8128, lr_0 = 9.1537e-04
Validation auc = 0.695775
Epoch 60
Validation auc = 0.700235
Epoch 61
Validation auc = 0.675352
Epoch 62
Validation auc = 0.719718
Epoch 63
Validation auc = 0.718310
Epoch 64
Loss = 3.7890e+00, PNorm = 51.0495, GNorm = 161.6754, lr_0 = 9.0754e-04
Validation auc = 0.693192
Epoch 65
Validation auc = 0.690376
Epoch 66
Validation auc = 0.693427
Epoch 67
Validation auc = 0.711502
Epoch 68
Validation auc = 0.708451
Epoch 69
Loss = 3.5375e+00, PNorm = 51.0724, GNorm = 313.1878, lr_0 = 8.9978e-04
Validation auc = 0.666432
Epoch 70
Validation auc = 0.630282
Epoch 71
Validation auc = 0.674648
Epoch 72
Validation auc = 0.710563
Epoch 73
Validation auc = 0.690845
Epoch 74
Loss = 2.8996e+00, PNorm = 51.0957, GNorm = 93.7283, lr_0 = 8.9209e-04
Validation auc = 0.650939
Epoch 75
Validation auc = 0.648122
Epoch 76
Validation auc = 0.669249
Epoch 77
Validation auc = 0.683568
Epoch 78
Validation auc = 0.698826
Epoch 79
Loss = 2.6405e+00, PNorm = 51.1170, GNorm = 112.7388, lr_0 = 8.8447e-04
Validation auc = 0.690845
Epoch 80
Validation auc = 0.709859
Epoch 81
Validation auc = 0.719014
Epoch 82
Validation auc = 0.692958
Epoch 83
Validation auc = 0.661502
Epoch 84
Loss = 3.4245e+00, PNorm = 51.1360, GNorm = 145.5555, lr_0 = 8.7691e-04
Validation auc = 0.704930
Epoch 85
Validation auc = 0.730047
Epoch 86
Validation auc = 0.714554
Epoch 87
Validation auc = 0.673709
Epoch 88
Validation auc = 0.654225
Epoch 89
Loss = 1.4107e+00, PNorm = 51.1541, GNorm = 80.6219, lr_0 = 8.6941e-04
Validation auc = 0.639671
Epoch 90
Validation auc = 0.663380
Epoch 91
Validation auc = 0.667840
Epoch 92
Validation auc = 0.685446
Epoch 93
Validation auc = 0.676526
Epoch 94
Loss = 2.2104e+00, PNorm = 51.1690, GNorm = 359.5884, lr_0 = 8.6198e-04
Validation auc = 0.655634
Epoch 95
Validation auc = 0.622300
Epoch 96
Validation auc = 0.653991
Epoch 97
Validation auc = 0.689671
Epoch 98
Validation auc = 0.711502
Epoch 99
Loss = 2.3160e+00, PNorm = 51.1860, GNorm = 128.6308, lr_0 = 8.5461e-04
Validation auc = 0.694366
Epoch 100
Validation auc = 0.673474
Epoch 101
Validation auc = 0.640376
Epoch 102
Validation auc = 0.613146
Epoch 103
Validation auc = 0.607746
Epoch 104
Loss = 9.9856e-01, PNorm = 51.2040, GNorm = 158.8769, lr_0 = 8.4730e-04
Validation auc = 0.618779
Epoch 105
Validation auc = 0.641080
Epoch 106
Validation auc = 0.671362
Epoch 107
Validation auc = 0.675587
Epoch 108
Validation auc = 0.671127
Epoch 109
Loss = 3.1290e+00, PNorm = 51.2210, GNorm = 445.8272, lr_0 = 8.4006e-04
Validation auc = 0.680047
Epoch 110
Validation auc = 0.637089
Epoch 111
Validation auc = 0.574178
Epoch 112
Validation auc = 0.591784
Epoch 113
Validation auc = 0.655399
Epoch 114
Loss = 1.3495e+00, PNorm = 51.2392, GNorm = 114.2069, lr_0 = 8.3288e-04
Validation auc = 0.657981
Epoch 115
Validation auc = 0.668310
Epoch 116
Validation auc = 0.650469
Epoch 117
Validation auc = 0.688967
Epoch 118
Validation auc = 0.704460
Epoch 119
Loss = 2.0453e+00, PNorm = 51.2551, GNorm = 107.0848, lr_0 = 8.2576e-04
Validation auc = 0.715962
Epoch 120
Validation auc = 0.655869
Epoch 121
Validation auc = 0.649531
Epoch 122
Validation auc = 0.712676
Epoch 123
Validation auc = 0.709155
Epoch 124
Loss = 2.1257e+00, PNorm = 51.2696, GNorm = 86.5785, lr_0 = 8.1870e-04
Validation auc = 0.703991
Epoch 125
Validation auc = 0.710798
Epoch 126
Validation auc = 0.706338
Epoch 127
Validation auc = 0.717371
Epoch 128
Validation auc = 0.726291
Epoch 129
Loss = 2.4430e+00, PNorm = 51.2839, GNorm = 103.3002, lr_0 = 8.1170e-04
Validation auc = 0.701643
Epoch 130
Validation auc = 0.713850
Epoch 131
Validation auc = 0.724178
Epoch 132
Validation auc = 0.709390
Epoch 133
Validation auc = 0.696948
Epoch 134
Loss = 5.9069e-01, PNorm = 51.2984, GNorm = 49.3052, lr_0 = 8.0476e-04
Validation auc = 0.680986
Epoch 135
Validation auc = 0.670188
Epoch 136
Validation auc = 0.676056
Epoch 137
Validation auc = 0.658920
Epoch 138
Validation auc = 0.657512
Epoch 139
Loss = 8.9144e-01, PNorm = 51.3112, GNorm = 48.6644, lr_0 = 7.9788e-04
Validation auc = 0.674413
Epoch 140
Validation auc = 0.666901
Epoch 141
Validation auc = 0.669014
Epoch 142
Validation auc = 0.679343
Epoch 143
Validation auc = 0.701878
Epoch 144
Loss = 3.3988e-01, PNorm = 51.3231, GNorm = 34.9892, lr_0 = 7.9106e-04
Validation auc = 0.715023
Epoch 145
Validation auc = 0.698826
Epoch 146
Validation auc = 0.684272
Epoch 147
Validation auc = 0.673239
Epoch 148
Validation auc = 0.664789
Epoch 149
Loss = 6.1904e-01, PNorm = 51.3339, GNorm = 46.7075, lr_0 = 7.8430e-04
Validation auc = 0.660798
Epoch 150
Validation auc = 0.664085
Epoch 151
Validation auc = 0.687089
Epoch 152
Validation auc = 0.683333
Epoch 153
Validation auc = 0.684038
Epoch 154
Loss = 7.8358e-01, PNorm = 51.3443, GNorm = 64.7026, lr_0 = 7.7759e-04
Validation auc = 0.676056
Epoch 155
Validation auc = 0.671362
Epoch 156
Validation auc = 0.644601
Epoch 157
Validation auc = 0.640141
Epoch 158
Validation auc = 0.652582
Epoch 159
Loss = 9.4933e-01, PNorm = 51.3546, GNorm = 43.3204, lr_0 = 7.7095e-04
Validation auc = 0.635211
Epoch 160
Validation auc = 0.601878
Epoch 161
Validation auc = 0.592958
Epoch 162
Validation auc = 0.584507
Epoch 163
Validation auc = 0.598592
Epoch 164
Loss = 1.0655e+00, PNorm = 51.3648, GNorm = 26.8785, lr_0 = 7.6436e-04
Validation auc = 0.646479
Epoch 165
Validation auc = 0.649296
Epoch 166
Validation auc = 0.652582
Epoch 167
Validation auc = 0.661268
Epoch 168
Validation auc = 0.658685
Epoch 169
Loss = 5.3981e-01, PNorm = 51.3748, GNorm = 53.7245, lr_0 = 7.5782e-04
Validation auc = 0.646948
Epoch 170
Validation auc = 0.647887
Epoch 171
Validation auc = 0.659624
Epoch 172
Validation auc = 0.689906
Epoch 173
Validation auc = 0.689906
Epoch 174
Loss = 7.6113e-01, PNorm = 51.3857, GNorm = 73.1131, lr_0 = 7.5134e-04
Validation auc = 0.668545
Epoch 175
Validation auc = 0.660329
Epoch 176
Validation auc = 0.665728
Epoch 177
Validation auc = 0.679108
Epoch 178
Validation auc = 0.685211
Epoch 179
Loss = 1.1172e+00, PNorm = 51.3987, GNorm = 90.8400, lr_0 = 7.4492e-04
Validation auc = 0.690376
Epoch 180
Validation auc = 0.693662
Epoch 181
Validation auc = 0.693662
Epoch 182
Validation auc = 0.675352
Epoch 183
Validation auc = 0.641080
Epoch 184
Loss = 8.3653e-01, PNorm = 51.4101, GNorm = 59.7859, lr_0 = 7.3855e-04
Validation auc = 0.640376
Epoch 185
Validation auc = 0.657981
Epoch 186
Validation auc = 0.683568
Epoch 187
Validation auc = 0.691080
Epoch 188
Validation auc = 0.690141
Epoch 189
Loss = 6.1396e-01, PNorm = 51.4197, GNorm = 24.8134, lr_0 = 7.3224e-04
Validation auc = 0.684507
Epoch 190
Validation auc = 0.683568
Epoch 191
Validation auc = 0.681690
Epoch 192
Validation auc = 0.681455
Epoch 193
Validation auc = 0.683333
Epoch 194
Loss = 4.5934e-01, PNorm = 51.4276, GNorm = 17.2797, lr_0 = 7.2598e-04
Validation auc = 0.682864
Epoch 195
Validation auc = 0.675352
Epoch 196
Validation auc = 0.673944
Epoch 197
Validation auc = 0.662207
Epoch 198
Validation auc = 0.648122
Epoch 199
Loss = 4.6129e-01, PNorm = 51.4345, GNorm = 45.0038, lr_0 = 7.1977e-04
Validation auc = 0.640376
Epoch 200
Validation auc = 0.628169
Epoch 201
Validation auc = 0.635681
Epoch 202
Validation auc = 0.638498
Epoch 203
Validation auc = 0.644131
Epoch 204
Loss = 3.2056e-01, PNorm = 51.4411, GNorm = 15.3990, lr_0 = 7.1362e-04
Validation auc = 0.651174
Epoch 205
Validation auc = 0.660563
Epoch 206
Validation auc = 0.668545
Epoch 207
Validation auc = 0.682629
Epoch 208
Validation auc = 0.688498
Epoch 209
Loss = 2.5959e-01, PNorm = 51.4477, GNorm = 16.1671, lr_0 = 7.0752e-04
Validation auc = 0.691549
Epoch 210
Validation auc = 0.693192
Epoch 211
Validation auc = 0.691080
Epoch 212
Validation auc = 0.674648
Epoch 213
Validation auc = 0.663850
Epoch 214
Loss = 8.0058e-01, PNorm = 51.4536, GNorm = 59.0818, lr_0 = 7.0147e-04
Validation auc = 0.652582
Epoch 215
Validation auc = 0.648357
Epoch 216
Validation auc = 0.640376
Epoch 217
Validation auc = 0.620657
Epoch 218
Validation auc = 0.601878
Epoch 219
Loss = 5.0883e-01, PNorm = 51.4594, GNorm = 40.0199, lr_0 = 6.9548e-04
Validation auc = 0.623005
Epoch 220
Validation auc = 0.650704
Epoch 221
Validation auc = 0.663850
Epoch 222
Validation auc = 0.686854
Epoch 223
Validation auc = 0.701408
Epoch 224
Loss = 4.6413e-01, PNorm = 51.4667, GNorm = 46.1307, lr_0 = 6.8953e-04
Validation auc = 0.700939
Epoch 225
Validation auc = 0.698592
Epoch 226
Validation auc = 0.703756
Epoch 227
Validation auc = 0.701408
Epoch 228
Validation auc = 0.692958
Epoch 229
Loss = 6.2064e-01, PNorm = 51.4738, GNorm = 44.1259, lr_0 = 6.8364e-04
Validation auc = 0.685915
Epoch 230
Validation auc = 0.683099
Epoch 231
Validation auc = 0.684977
Epoch 232
Validation auc = 0.681221
Epoch 233
Validation auc = 0.663850
Epoch 234
Loss = 7.4680e-01, PNorm = 51.4801, GNorm = 59.4935, lr_0 = 6.7779e-04
Validation auc = 0.652113
Epoch 235
Validation auc = 0.643662
Epoch 236
Validation auc = 0.650235
Epoch 237
Validation auc = 0.662441
Epoch 238
Validation auc = 0.674648
Epoch 239
Loss = 2.6347e-01, PNorm = 51.4864, GNorm = 28.0309, lr_0 = 6.7200e-04
Validation auc = 0.664319
Epoch 240
Validation auc = 0.669953
Epoch 241
Validation auc = 0.677934
Epoch 242
Validation auc = 0.684507
Epoch 243
Validation auc = 0.694366
Epoch 244
Loss = 2.9109e-01, PNorm = 51.4927, GNorm = 25.6765, lr_0 = 6.6625e-04
Validation auc = 0.692958
Epoch 245
Validation auc = 0.691549
Epoch 246
Validation auc = 0.686385
Epoch 247
Validation auc = 0.686385
Epoch 248
Validation auc = 0.683099
Epoch 249
Loss = 2.7646e-01, PNorm = 51.4981, GNorm = 14.1128, lr_0 = 6.6056e-04
Validation auc = 0.678873
Epoch 250
Validation auc = 0.670892
Epoch 251
Validation auc = 0.668075
Epoch 252
Validation auc = 0.669014
Epoch 253
Validation auc = 0.651643
Epoch 254
Loss = 1.0799e-01, PNorm = 51.5029, GNorm = 8.4042, lr_0 = 6.5491e-04
Validation auc = 0.635681
Epoch 255
Validation auc = 0.625822
Epoch 256
Validation auc = 0.621127
Epoch 257
Validation auc = 0.621596
Epoch 258
Validation auc = 0.622066
Epoch 259
Loss = 2.7272e-01, PNorm = 51.5079, GNorm = 16.5502, lr_0 = 6.4931e-04
Validation auc = 0.628169
Epoch 260
Validation auc = 0.636150
Epoch 261
Validation auc = 0.647887
Epoch 262
Validation auc = 0.657277
Epoch 263
Validation auc = 0.675117
Epoch 264
Loss = 3.4857e-01, PNorm = 51.5135, GNorm = 32.8127, lr_0 = 6.4376e-04
Validation auc = 0.686385
Epoch 265
Validation auc = 0.694836
Epoch 266
Validation auc = 0.696244
Epoch 267
Validation auc = 0.689202
Epoch 268
Validation auc = 0.684507
Epoch 269
Loss = 5.3146e-01, PNorm = 51.5191, GNorm = 39.2550, lr_0 = 6.3826e-04
Validation auc = 0.685446
Epoch 270
Validation auc = 0.696714
Epoch 271
Validation auc = 0.700939
Epoch 272
Validation auc = 0.702347
Epoch 273
Validation auc = 0.699531
Epoch 274
Loss = 7.7285e-01, PNorm = 51.5246, GNorm = 60.4043, lr_0 = 6.3280e-04
Validation auc = 0.695305
Epoch 275
Validation auc = 0.689202
Epoch 276
Validation auc = 0.690141
Epoch 277
Validation auc = 0.687793
Epoch 278
Validation auc = 0.687793
Epoch 279
Loss = 1.7862e-01, PNorm = 51.5302, GNorm = 20.6065, lr_0 = 6.2739e-04
Validation auc = 0.687793
Epoch 280
Validation auc = 0.684977
Epoch 281
Validation auc = 0.695775
Epoch 282
Validation auc = 0.698592
Epoch 283
Validation auc = 0.699061
Epoch 284
Loss = 2.0401e-01, PNorm = 51.5361, GNorm = 30.6501, lr_0 = 6.2203e-04
Validation auc = 0.697653
Epoch 285
Validation auc = 0.700939
Epoch 286
Validation auc = 0.696244
Epoch 287
Validation auc = 0.692958
Epoch 288
Validation auc = 0.688732
Epoch 289
Loss = 2.4988e-01, PNorm = 51.5422, GNorm = 21.3199, lr_0 = 6.1671e-04
Validation auc = 0.689202
Epoch 290
Validation auc = 0.691549
Epoch 291
Validation auc = 0.690610
Epoch 292
Validation auc = 0.694366
Epoch 293
Validation auc = 0.699531
Epoch 294
Loss = 1.8101e-01, PNorm = 51.5477, GNorm = 13.6324, lr_0 = 6.1144e-04
Validation auc = 0.693427
Epoch 295
Validation auc = 0.700939
Epoch 296
Validation auc = 0.705634
Epoch 297
Validation auc = 0.709859
Epoch 298
Validation auc = 0.702817
Epoch 299
Loss = 2.8902e-01, PNorm = 51.5532, GNorm = 47.0296, lr_0 = 6.0621e-04
Validation auc = 0.697183
Model 0 best validation auc = 0.765493 on epoch 28
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.732691
Ensemble test auc = 0.732691
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Validation auc = 0.786385
Epoch 1
Validation auc = 0.664554
Epoch 2
Validation auc = 0.767840
Epoch 3
Validation auc = 0.640376
Epoch 4
Loss = 5.7387e+01, PNorm = 50.2690, GNorm = 911.4391, lr_0 = 6.5000e-04
Validation auc = 0.694131
Epoch 5
Validation auc = 0.590376
Epoch 6
Validation auc = 0.679812
Epoch 7
Validation auc = 0.647653
Epoch 8
Validation auc = 0.641315
Epoch 9
Loss = 2.8322e+01, PNorm = 50.4063, GNorm = 280.0171, lr_0 = 9.9743e-04
Validation auc = 0.710798
Epoch 10
Validation auc = 0.683099
Epoch 11
Validation auc = 0.636150
Epoch 12
Validation auc = 0.687324
Epoch 13
Validation auc = 0.695305
Epoch 14
Loss = 1.2944e+01, PNorm = 50.5580, GNorm = 666.8020, lr_0 = 9.8890e-04
Validation auc = 0.708451
Epoch 15
Validation auc = 0.619718
Epoch 16
Validation auc = 0.646244
Epoch 17
Validation auc = 0.668545
Epoch 18
Validation auc = 0.654930
Epoch 19
Loss = 2.0657e+01, PNorm = 50.6701, GNorm = 843.1420, lr_0 = 9.8045e-04
Validation auc = 0.712207
Epoch 20
Validation auc = 0.631690
Epoch 21
Validation auc = 0.699296
Epoch 22
Validation auc = 0.718310
Epoch 23
Validation auc = 0.667136
Epoch 24
Loss = 2.4136e+01, PNorm = 50.7519, GNorm = 1143.8690, lr_0 = 9.7207e-04
Validation auc = 0.604930
Epoch 25
Validation auc = 0.678169
Epoch 26
Validation auc = 0.616667
Epoch 27
Validation auc = 0.630282
Epoch 28
Validation auc = 0.677934
Epoch 29
Loss = 2.8453e+01, PNorm = 50.8147, GNorm = 174.7248, lr_0 = 9.6376e-04
Validation auc = 0.645540
Epoch 30
Validation auc = 0.720188
Epoch 31
Validation auc = 0.707746
Epoch 32
Validation auc = 0.691784
Epoch 33
Validation auc = 0.756808
Epoch 34
Loss = 1.4487e+01, PNorm = 50.8701, GNorm = 601.1397, lr_0 = 9.5552e-04
Validation auc = 0.659859
Epoch 35
Validation auc = 0.733803
Epoch 36
Validation auc = 0.683568
Epoch 37
Validation auc = 0.745070
Epoch 38
Validation auc = 0.762207
Epoch 39
Loss = 1.1014e+01, PNorm = 50.9175, GNorm = 286.2495, lr_0 = 9.4735e-04
Validation auc = 0.705634
Epoch 40
Validation auc = 0.661502
Epoch 41
Validation auc = 0.681221
Epoch 42
Validation auc = 0.743897
Epoch 43
Validation auc = 0.700000
Epoch 44
Loss = 7.3976e+00, PNorm = 50.9535, GNorm = 261.4594, lr_0 = 9.3925e-04
Validation auc = 0.663850
Epoch 45
Validation auc = 0.653756
Epoch 46
Validation auc = 0.669953
Epoch 47
Validation auc = 0.678404
Epoch 48
Validation auc = 0.673709
Epoch 49
Loss = 8.7454e+00, PNorm = 50.9841, GNorm = 580.0929, lr_0 = 9.3122e-04
Validation auc = 0.659624
Epoch 50
Validation auc = 0.713146
Epoch 51
Validation auc = 0.712207
Epoch 52
Validation auc = 0.650939
Epoch 53
Validation auc = 0.728873
Epoch 54
Loss = 7.5028e+00, PNorm = 51.0144, GNorm = 352.5559, lr_0 = 9.2326e-04
Validation auc = 0.732629
Epoch 55
Validation auc = 0.653286
Epoch 56
Validation auc = 0.636854
Epoch 57
Validation auc = 0.679343
Epoch 58
Validation auc = 0.684742
Epoch 59
Loss = 1.0844e+01, PNorm = 51.0433, GNorm = 319.5349, lr_0 = 9.1537e-04
Validation auc = 0.644601
Epoch 60
Validation auc = 0.644601
Epoch 61
Validation auc = 0.639202
Epoch 62
Validation auc = 0.657746
Epoch 63
Validation auc = 0.661502
Epoch 64
Loss = 9.0626e+00, PNorm = 51.0676, GNorm = 423.9821, lr_0 = 9.0754e-04
Validation auc = 0.697418
Epoch 65
Validation auc = 0.683099
Epoch 66
Validation auc = 0.683099
Epoch 67
Validation auc = 0.662441
Epoch 68
Validation auc = 0.558451
Epoch 69
Loss = 9.1847e+00, PNorm = 51.0926, GNorm = 386.4442, lr_0 = 8.9978e-04
Validation auc = 0.601643
Epoch 70
Validation auc = 0.545070
Epoch 71
Validation auc = 0.653286
Epoch 72
Validation auc = 0.650704
Epoch 73
Validation auc = 0.666197
Epoch 74
Loss = 3.4169e+00, PNorm = 51.1165, GNorm = 116.2947, lr_0 = 8.9209e-04
Validation auc = 0.703052
Epoch 75
Validation auc = 0.704460
Epoch 76
Validation auc = 0.709390
Epoch 77
Validation auc = 0.680047
Epoch 78
Validation auc = 0.644836
Epoch 79
Loss = 4.2549e+00, PNorm = 51.1394, GNorm = 434.8195, lr_0 = 8.8447e-04
Validation auc = 0.629343
Epoch 80
Validation auc = 0.592723
Epoch 81
Validation auc = 0.551878
Epoch 82
Validation auc = 0.560798
Epoch 83
Validation auc = 0.552113
Epoch 84
Loss = 3.1603e+00, PNorm = 51.1616, GNorm = 106.5974, lr_0 = 8.7691e-04
Validation auc = 0.553756
Epoch 85
Validation auc = 0.567840
Epoch 86
Validation auc = 0.594836
Epoch 87
Validation auc = 0.626056
Epoch 88
Validation auc = 0.629812
Epoch 89
Loss = 1.7203e+00, PNorm = 51.1828, GNorm = 117.9497, lr_0 = 8.6941e-04
Validation auc = 0.634038
Epoch 90
Validation auc = 0.625117
Epoch 91
Validation auc = 0.649765
Epoch 92
Validation auc = 0.662676
Epoch 93
Validation auc = 0.636620
Epoch 94
Loss = 1.6600e+00, PNorm = 51.2017, GNorm = 195.0315, lr_0 = 8.6198e-04
Validation auc = 0.623474
Epoch 95
Validation auc = 0.572300
Epoch 96
Validation auc = 0.593192
Epoch 97
Validation auc = 0.608216
Epoch 98
Validation auc = 0.639437
Epoch 99
Loss = 2.4535e+00, PNorm = 51.2202, GNorm = 149.9480, lr_0 = 8.5461e-04
Validation auc = 0.652817
Epoch 100
Validation auc = 0.695540
Epoch 101
Validation auc = 0.695775
Epoch 102
Validation auc = 0.683333
Epoch 103
Validation auc = 0.668779
Epoch 104
Loss = 3.3679e+00, PNorm = 51.2395, GNorm = 309.5188, lr_0 = 8.4730e-04
Validation auc = 0.658685
Epoch 105
Validation auc = 0.662676
Epoch 106
Validation auc = 0.648122
Epoch 107
Validation auc = 0.642254
Epoch 108
Validation auc = 0.629108
Epoch 109
Loss = 3.3814e+00, PNorm = 51.2589, GNorm = 148.6545, lr_0 = 8.4006e-04
Validation auc = 0.647887
Epoch 110
Validation auc = 0.640845
Epoch 111
Validation auc = 0.615258
Epoch 112
Validation auc = 0.625352
Epoch 113
Validation auc = 0.615962
Epoch 114
Loss = 1.6164e+00, PNorm = 51.2759, GNorm = 81.1820, lr_0 = 8.3288e-04
Validation auc = 0.596948
Epoch 115
Validation auc = 0.630751
Epoch 116
Validation auc = 0.674413
Epoch 117
Validation auc = 0.677465
Epoch 118
Validation auc = 0.687793
Epoch 119
Loss = 1.1831e+00, PNorm = 51.2910, GNorm = 122.9299, lr_0 = 8.2576e-04
Validation auc = 0.708216
Epoch 120
Validation auc = 0.680282
Epoch 121
Validation auc = 0.669484
Epoch 122
Validation auc = 0.692254
Epoch 123
Validation auc = 0.730516
Epoch 124
Loss = 1.5049e+00, PNorm = 51.3052, GNorm = 184.8349, lr_0 = 8.1870e-04
Validation auc = 0.719249
Epoch 125
Validation auc = 0.719014
Epoch 126
Validation auc = 0.666901
Epoch 127
Validation auc = 0.592019
Epoch 128
Validation auc = 0.587324
Epoch 129
Loss = 1.7408e+00, PNorm = 51.3190, GNorm = 204.2463, lr_0 = 8.1170e-04
Validation auc = 0.589437
Epoch 130
Validation auc = 0.648357
Epoch 131
Validation auc = 0.671362
Epoch 132
Validation auc = 0.672535
Epoch 133
Validation auc = 0.666197
Epoch 134
Loss = 1.2184e+00, PNorm = 51.3321, GNorm = 75.6194, lr_0 = 8.0476e-04
Validation auc = 0.665258
Epoch 135
Validation auc = 0.665493
Epoch 136
Validation auc = 0.665962
Epoch 137
Validation auc = 0.654695
Epoch 138
Validation auc = 0.645305
Epoch 139
Loss = 1.4747e+00, PNorm = 51.3446, GNorm = 96.3346, lr_0 = 7.9788e-04
Validation auc = 0.671596
Epoch 140
Validation auc = 0.663146
Epoch 141
Validation auc = 0.671362
Epoch 142
Validation auc = 0.661033
Epoch 143
Validation auc = 0.671127
Epoch 144
Loss = 9.9330e-01, PNorm = 51.3568, GNorm = 60.0462, lr_0 = 7.9106e-04
Validation auc = 0.696948
Epoch 145
Validation auc = 0.708216
Epoch 146
Validation auc = 0.712911
Epoch 147
Validation auc = 0.718075
Epoch 148
Validation auc = 0.701643
Epoch 149
Loss = 1.9525e+00, PNorm = 51.3686, GNorm = 71.6962, lr_0 = 7.8430e-04
Validation auc = 0.691315
Epoch 150
Validation auc = 0.681925
Epoch 151
Validation auc = 0.666432
Epoch 152
Validation auc = 0.660798
Epoch 153
Validation auc = 0.642019
Epoch 154
Loss = 5.4591e-01, PNorm = 51.3799, GNorm = 54.7040, lr_0 = 7.7759e-04
Validation auc = 0.646714
Epoch 155
Validation auc = 0.640376
Epoch 156
Validation auc = 0.633803
Epoch 157
Validation auc = 0.634742
Epoch 158
Validation auc = 0.634272
Epoch 159
Loss = 1.3559e+00, PNorm = 51.3924, GNorm = 87.3374, lr_0 = 7.7095e-04
Validation auc = 0.612676
Epoch 160
Validation auc = 0.626761
Epoch 161
Validation auc = 0.638967
Epoch 162
Validation auc = 0.657746
Epoch 163
Validation auc = 0.657746
Epoch 164
Loss = 1.0234e+00, PNorm = 51.4052, GNorm = 98.1262, lr_0 = 7.6436e-04
Validation auc = 0.679343
Epoch 165
Validation auc = 0.696948
Epoch 166
Validation auc = 0.688263
Epoch 167
Validation auc = 0.701643
Epoch 168
Validation auc = 0.683568
Epoch 169
Loss = 6.0338e-01, PNorm = 51.4162, GNorm = 45.7461, lr_0 = 7.5782e-04
Validation auc = 0.673239
Epoch 170
Validation auc = 0.679577
Epoch 171
Validation auc = 0.644601
Epoch 172
Validation auc = 0.630282
Epoch 173
Validation auc = 0.651174
Epoch 174
Loss = 3.7981e-01, PNorm = 51.4264, GNorm = 26.9358, lr_0 = 7.5134e-04
Validation auc = 0.664319
Epoch 175
Validation auc = 0.668310
Epoch 176
Validation auc = 0.668310
Epoch 177
Validation auc = 0.658685
Epoch 178
Validation auc = 0.647653
Epoch 179
Loss = 4.5130e-01, PNorm = 51.4372, GNorm = 29.3404, lr_0 = 7.4492e-04
Validation auc = 0.624178
Epoch 180
Validation auc = 0.607277
Epoch 181
Validation auc = 0.592254
Epoch 182
Validation auc = 0.569014
Epoch 183
Validation auc = 0.542723
Epoch 184
Loss = 9.8700e-01, PNorm = 51.4484, GNorm = 195.7780, lr_0 = 7.3855e-04
Validation auc = 0.533803
Epoch 185
Validation auc = 0.561268
Epoch 186
Validation auc = 0.584038
Epoch 187
Validation auc = 0.626761
Epoch 188
Validation auc = 0.661502
Epoch 189
Loss = 7.2480e-01, PNorm = 51.4583, GNorm = 32.3126, lr_0 = 7.3224e-04
Validation auc = 0.655869
Epoch 190
Validation auc = 0.662911
Epoch 191
Validation auc = 0.675822
Epoch 192
Validation auc = 0.685681
Epoch 193
Validation auc = 0.692254
Epoch 194
Loss = 6.3727e-01, PNorm = 51.4686, GNorm = 43.0970, lr_0 = 7.2598e-04
Validation auc = 0.685915
Epoch 195
Validation auc = 0.661502
Epoch 196
Validation auc = 0.670423
Epoch 197
Validation auc = 0.683099
Epoch 198
Validation auc = 0.690610
Epoch 199
Loss = 1.1121e+00, PNorm = 51.4791, GNorm = 100.1823, lr_0 = 7.1977e-04
Validation auc = 0.675822
Epoch 200
Validation auc = 0.639906
Epoch 201
Validation auc = 0.631925
Epoch 202
Validation auc = 0.616432
Epoch 203
Validation auc = 0.603756
Epoch 204
Loss = 5.5000e-01, PNorm = 51.4884, GNorm = 29.5587, lr_0 = 7.1362e-04
Validation auc = 0.584977
Epoch 205
Validation auc = 0.567840
Epoch 206
Validation auc = 0.574178
Epoch 207
Validation auc = 0.586385
Epoch 208
Validation auc = 0.588263
Epoch 209
Loss = 8.1852e-01, PNorm = 51.4974, GNorm = 61.9092, lr_0 = 7.0752e-04
Validation auc = 0.593897
Epoch 210
Validation auc = 0.604225
Epoch 211
Validation auc = 0.620188
Epoch 212
Validation auc = 0.661033
Epoch 213
Validation auc = 0.668545
Epoch 214
Loss = 1.1287e+00, PNorm = 51.5054, GNorm = 40.3711, lr_0 = 7.0147e-04
Validation auc = 0.669014
Epoch 215
Validation auc = 0.648826
Epoch 216
Validation auc = 0.641315
Epoch 217
Validation auc = 0.642723
Epoch 218
Validation auc = 0.659624
Epoch 219
Loss = 3.2326e-01, PNorm = 51.5127, GNorm = 11.0846, lr_0 = 6.9548e-04
Validation auc = 0.656573
Epoch 220
Validation auc = 0.649061
Epoch 221
Validation auc = 0.663850
Epoch 222
Validation auc = 0.669014
Epoch 223
Validation auc = 0.670892
Epoch 224
Loss = 1.0352e+00, PNorm = 51.5199, GNorm = 134.5446, lr_0 = 6.8953e-04
Validation auc = 0.676526
Epoch 225
Validation auc = 0.679108
Epoch 226
Validation auc = 0.668545
Epoch 227
Validation auc = 0.671831
Epoch 228
Validation auc = 0.692488
Epoch 229
Loss = 6.0017e-01, PNorm = 51.5274, GNorm = 43.2739, lr_0 = 6.8364e-04
Validation auc = 0.676291
Epoch 230
Validation auc = 0.663615
Epoch 231
Validation auc = 0.668779
Epoch 232
Validation auc = 0.669249
Epoch 233
Validation auc = 0.688732
Epoch 234
Loss = 2.5684e-01, PNorm = 51.5355, GNorm = 56.0380, lr_0 = 6.7779e-04
Validation auc = 0.699531
Epoch 235
Validation auc = 0.699531
Epoch 236
Validation auc = 0.704695
Epoch 237
Validation auc = 0.703756
Epoch 238
Validation auc = 0.709859
Epoch 239
Loss = 3.8201e-01, PNorm = 51.5418, GNorm = 39.9893, lr_0 = 6.7200e-04
Validation auc = 0.712207
Epoch 240
Validation auc = 0.702347
Epoch 241
Validation auc = 0.677934
Epoch 242
Validation auc = 0.652582
Epoch 243
Validation auc = 0.632864
Epoch 244
Loss = 5.0592e-01, PNorm = 51.5479, GNorm = 58.2696, lr_0 = 6.6625e-04
Validation auc = 0.617840
Epoch 245
Validation auc = 0.628169
Epoch 246
Validation auc = 0.634272
Epoch 247
Validation auc = 0.663380
Epoch 248
Validation auc = 0.665258
Epoch 249
Loss = 1.2460e-01, PNorm = 51.5542, GNorm = 13.5000, lr_0 = 6.6056e-04
Validation auc = 0.650235
Epoch 250
Validation auc = 0.651643
Epoch 251
Validation auc = 0.657746
Epoch 252
Validation auc = 0.652113
Epoch 253
Validation auc = 0.661502
Epoch 254
Loss = 1.9792e-01, PNorm = 51.5594, GNorm = 10.7308, lr_0 = 6.5491e-04
Validation auc = 0.689202
Epoch 255
Validation auc = 0.700939
Epoch 256
Validation auc = 0.701878
Epoch 257
Validation auc = 0.698122
Epoch 258
Validation auc = 0.699061
Epoch 259
Loss = 5.0424e-01, PNorm = 51.5645, GNorm = 48.6380, lr_0 = 6.4931e-04
Validation auc = 0.699061
Epoch 260
Validation auc = 0.700469
Epoch 261
Validation auc = 0.702817
Epoch 262
Validation auc = 0.705164
Epoch 263
Validation auc = 0.692488
Epoch 264
Loss = 4.5504e-01, PNorm = 51.5705, GNorm = 52.0696, lr_0 = 6.4376e-04
Validation auc = 0.678873
Epoch 265
Validation auc = 0.652113
Epoch 266
Validation auc = 0.645540
Epoch 267
Validation auc = 0.663380
Epoch 268
Validation auc = 0.665728
Epoch 269
Loss = 4.6546e-01, PNorm = 51.5773, GNorm = 41.1832, lr_0 = 6.3826e-04
Validation auc = 0.663380
Epoch 270
Validation auc = 0.665258
Epoch 271
Validation auc = 0.663850
Epoch 272
Validation auc = 0.655399
Epoch 273
Validation auc = 0.657277
Epoch 274
Loss = 3.7715e-01, PNorm = 51.5852, GNorm = 46.5833, lr_0 = 6.3280e-04
Validation auc = 0.653052
Epoch 275
Validation auc = 0.651643
Epoch 276
Validation auc = 0.646948
Epoch 277
Validation auc = 0.650235
Epoch 278
Validation auc = 0.661502
Epoch 279
Loss = 4.3449e-01, PNorm = 51.5927, GNorm = 23.9868, lr_0 = 6.2739e-04
Validation auc = 0.660563
Epoch 280
Validation auc = 0.671831
Epoch 281
Validation auc = 0.657277
Epoch 282
Validation auc = 0.640376
Epoch 283
Validation auc = 0.631925
Epoch 284
Loss = 8.1064e-01, PNorm = 51.6005, GNorm = 65.7991, lr_0 = 6.2203e-04
Validation auc = 0.636150
Epoch 285
Validation auc = 0.657277
Epoch 286
Validation auc = 0.677934
Epoch 287
Validation auc = 0.711737
Epoch 288
Validation auc = 0.708920
Epoch 289
Loss = 5.1943e-01, PNorm = 51.6084, GNorm = 34.0702, lr_0 = 6.1671e-04
Validation auc = 0.707981
Epoch 290
Validation auc = 0.693427
Epoch 291
Validation auc = 0.667606
Epoch 292
Validation auc = 0.658216
Epoch 293
Validation auc = 0.641315
Epoch 294
Loss = 2.7370e-01, PNorm = 51.6163, GNorm = 31.6965, lr_0 = 6.1144e-04
Validation auc = 0.628169
Epoch 295
Validation auc = 0.624413
Epoch 296
Validation auc = 0.626291
Epoch 297
Validation auc = 0.637559
Epoch 298
Validation auc = 0.645540
Epoch 299
Loss = 4.6062e-01, PNorm = 51.6239, GNorm = 20.6982, lr_0 = 6.0621e-04
Validation auc = 0.652113
Model 0 best validation auc = 0.786385 on epoch 0
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.840458
Ensemble test auc = 0.840458
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Warning: Found a task with predictions all 0s or all 1s
Validation auc = nan
Epoch 1
Validation auc = 0.659624
Epoch 2
Validation auc = 0.663850
Epoch 3
Validation auc = 0.682160
Epoch 4
Loss = 4.4528e+01, PNorm = 50.2712, GNorm = 1423.2013, lr_0 = 6.5000e-04
Validation auc = 0.749296
Epoch 5
Validation auc = 0.589671
Epoch 6
Validation auc = 0.652582
Epoch 7
Validation auc = 0.691080
Epoch 8
Validation auc = 0.570423
Epoch 9
Loss = 5.9423e+01, PNorm = 50.3995, GNorm = 1250.8713, lr_0 = 9.9743e-04
Validation auc = 0.659624
Epoch 10
Validation auc = 0.606573
Epoch 11
Validation auc = 0.708920
Epoch 12
Validation auc = 0.712207
Epoch 13
Validation auc = 0.633333
Epoch 14
Loss = 4.6250e+01, PNorm = 50.5442, GNorm = 1113.1235, lr_0 = 9.8890e-04
Validation auc = 0.722066
Epoch 15
Validation auc = 0.730986
Epoch 16
Validation auc = 0.649765
Epoch 17
Validation auc = 0.702817
Epoch 18
Validation auc = 0.669249
Epoch 19
Loss = 1.2966e+01, PNorm = 50.6541, GNorm = 757.4574, lr_0 = 9.8045e-04
Validation auc = 0.665493
Epoch 20
Validation auc = 0.738498
Epoch 21
Validation auc = 0.686385
Epoch 22
Validation auc = 0.695540
Epoch 23
Validation auc = 0.646479
Epoch 24
Loss = 3.6159e+01, PNorm = 50.7309, GNorm = 752.3178, lr_0 = 9.7207e-04
Validation auc = 0.496714
Epoch 25
Validation auc = 0.564319
Epoch 26
Validation auc = 0.762911
Epoch 27
Validation auc = 0.730282
Epoch 28
Validation auc = 0.651643
Epoch 29
Loss = 2.0917e+01, PNorm = 50.7952, GNorm = 573.7403, lr_0 = 9.6376e-04
Validation auc = 0.674178
Epoch 30
Validation auc = 0.712441
Epoch 31
Validation auc = 0.706103
Epoch 32
Validation auc = 0.535211
Epoch 33
Validation auc = 0.630516
Epoch 34
Loss = 1.2532e+01, PNorm = 50.8444, GNorm = 667.4825, lr_0 = 9.5552e-04
Validation auc = 0.691080
Epoch 35
Validation auc = 0.694366
Epoch 36
Validation auc = 0.680047
Epoch 37
Validation auc = 0.651174
Epoch 38
Validation auc = 0.707042
Epoch 39
Loss = 7.0304e+00, PNorm = 50.8820, GNorm = 225.4622, lr_0 = 9.4735e-04
Validation auc = 0.655869
Epoch 40
Validation auc = 0.660094
Epoch 41
Validation auc = 0.685446
Epoch 42
Validation auc = 0.690376
Epoch 43
Validation auc = 0.662676
Epoch 44
Loss = 6.9471e+00, PNorm = 50.9140, GNorm = 262.6468, lr_0 = 9.3925e-04
Validation auc = 0.650235
Epoch 45
Validation auc = 0.695305
Epoch 46
Validation auc = 0.654460
Epoch 47
Validation auc = 0.679812
Epoch 48
Validation auc = 0.728169
Epoch 49
Loss = 8.6710e+00, PNorm = 50.9432, GNorm = 461.4493, lr_0 = 9.3122e-04
Validation auc = 0.732160
Epoch 50
Validation auc = 0.666667
Epoch 51
Validation auc = 0.668779
Epoch 52
Validation auc = 0.691549
Epoch 53
Validation auc = 0.689202
Epoch 54
Loss = 7.0570e+00, PNorm = 50.9694, GNorm = 346.1177, lr_0 = 9.2326e-04
Validation auc = 0.623239
Epoch 55
Validation auc = 0.639906
Epoch 56
Validation auc = 0.690610
Epoch 57
Validation auc = 0.680751
Epoch 58
Validation auc = 0.686854
Epoch 59
Loss = 4.5688e+00, PNorm = 50.9974, GNorm = 171.3807, lr_0 = 9.1537e-04
Validation auc = 0.644601
Epoch 60
Validation auc = 0.634272
Epoch 61
Validation auc = 0.715023
Epoch 62
Validation auc = 0.664319
Epoch 63
Validation auc = 0.662911
Epoch 64
Loss = 4.1675e+00, PNorm = 51.0245, GNorm = 123.9255, lr_0 = 9.0754e-04
Validation auc = 0.638498
Epoch 65
Validation auc = 0.643427
Epoch 66
Validation auc = 0.648826
Epoch 67
Validation auc = 0.614789
Epoch 68
Validation auc = 0.601408
Epoch 69
Loss = 4.2388e+00, PNorm = 51.0504, GNorm = 137.4892, lr_0 = 8.9978e-04
Validation auc = 0.645305
Epoch 70
Validation auc = 0.652113
Epoch 71
Validation auc = 0.638967
Epoch 72
Validation auc = 0.668075
Epoch 73
Validation auc = 0.621831
Epoch 74
Loss = 3.8771e+00, PNorm = 51.0726, GNorm = 172.5691, lr_0 = 8.9209e-04
Validation auc = 0.646948
Epoch 75
Validation auc = 0.688263
Epoch 76
Validation auc = 0.656103
Epoch 77
Validation auc = 0.676056
Epoch 78
Validation auc = 0.693662
Epoch 79
Loss = 1.7359e+00, PNorm = 51.0926, GNorm = 121.7276, lr_0 = 8.8447e-04
Validation auc = 0.684977
Epoch 80
Validation auc = 0.688967
Epoch 81
Validation auc = 0.702347
Epoch 82
Validation auc = 0.688028
Epoch 83
Validation auc = 0.687793
Epoch 84
Loss = 1.0669e+00, PNorm = 51.1104, GNorm = 135.0587, lr_0 = 8.7691e-04
Validation auc = 0.687089
Epoch 85
Validation auc = 0.665258
Epoch 86
Validation auc = 0.645775
Epoch 87
Validation auc = 0.638263
Epoch 88
Validation auc = 0.632864
Epoch 89
Loss = 2.3145e+00, PNorm = 51.1271, GNorm = 204.8205, lr_0 = 8.6941e-04
Validation auc = 0.609155
Epoch 90
Validation auc = 0.627465
Epoch 91
Validation auc = 0.634038
Epoch 92
Validation auc = 0.658685
Epoch 93
Validation auc = 0.674648
Epoch 94
Loss = 1.8929e+00, PNorm = 51.1447, GNorm = 111.2207, lr_0 = 8.6198e-04
Validation auc = 0.675352
Epoch 95
Validation auc = 0.665962
Epoch 96
Validation auc = 0.661737
Epoch 97
Validation auc = 0.659624
Epoch 98
Validation auc = 0.662911
Epoch 99
Loss = 1.2752e+00, PNorm = 51.1617, GNorm = 123.9554, lr_0 = 8.5461e-04
Validation auc = 0.673474
Epoch 100
Validation auc = 0.646479
Epoch 101
Validation auc = 0.626291
Epoch 102
Validation auc = 0.675587
Epoch 103
Validation auc = 0.694601
Epoch 104
Loss = 1.5174e+00, PNorm = 51.1764, GNorm = 106.2498, lr_0 = 8.4730e-04
Validation auc = 0.684507
Epoch 105
Validation auc = 0.696948
Epoch 106
Validation auc = 0.699531
Epoch 107
Validation auc = 0.670892
Epoch 108
Validation auc = 0.660094
Epoch 109
Loss = 6.7824e-01, PNorm = 51.1914, GNorm = 52.2110, lr_0 = 8.4006e-04
Validation auc = 0.670657
Epoch 110
Validation auc = 0.676761
Epoch 111
Validation auc = 0.670423
Epoch 112
Validation auc = 0.629108
Epoch 113
Validation auc = 0.603756
Epoch 114
Loss = 1.4652e+00, PNorm = 51.2066, GNorm = 85.3951, lr_0 = 8.3288e-04
Validation auc = 0.589671
Epoch 115
Validation auc = 0.630516
Epoch 116
Validation auc = 0.657746
Epoch 117
Validation auc = 0.652582
Epoch 118
Validation auc = 0.678404
Epoch 119
Loss = 1.5726e+00, PNorm = 51.2238, GNorm = 64.3103, lr_0 = 8.2576e-04
Validation auc = 0.706808
Epoch 120
Validation auc = 0.682864
Epoch 121
Validation auc = 0.676526
Epoch 122
Validation auc = 0.665962
Epoch 123
Validation auc = 0.673005
Epoch 124
Loss = 2.1552e+00, PNorm = 51.2405, GNorm = 196.1750, lr_0 = 8.1870e-04
Validation auc = 0.655164
Epoch 125
Validation auc = 0.645305
Epoch 126
Validation auc = 0.630047
Epoch 127
Validation auc = 0.629108
Epoch 128
Validation auc = 0.629577
Epoch 129
Loss = 5.2417e-01, PNorm = 51.2559, GNorm = 60.0537, lr_0 = 8.1170e-04
Validation auc = 0.628873
Epoch 130
Validation auc = 0.643192
Epoch 131
Validation auc = 0.648357
Epoch 132
Validation auc = 0.653052
Epoch 133
Validation auc = 0.658685
Epoch 134
Loss = 7.7419e-01, PNorm = 51.2680, GNorm = 88.2789, lr_0 = 8.0476e-04
Validation auc = 0.661033
Epoch 135
Validation auc = 0.660563
Epoch 136
Validation auc = 0.641549
Epoch 137
Validation auc = 0.643192
Epoch 138
Validation auc = 0.654225
Epoch 139
Loss = 7.5855e-01, PNorm = 51.2784, GNorm = 54.6459, lr_0 = 7.9788e-04
Validation auc = 0.645070
Epoch 140
Validation auc = 0.626291
Epoch 141
Validation auc = 0.601643
Epoch 142
Validation auc = 0.603052
Epoch 143
Validation auc = 0.616432
Epoch 144
Loss = 7.5980e-01, PNorm = 51.2889, GNorm = 60.7509, lr_0 = 7.9106e-04
Validation auc = 0.635681
Epoch 145
Validation auc = 0.634742
Epoch 146
Validation auc = 0.651643
Epoch 147
Validation auc = 0.644366
Epoch 148
Validation auc = 0.641080
Epoch 149
Loss = 1.4191e+00, PNorm = 51.2990, GNorm = 227.2697, lr_0 = 7.8430e-04
Validation auc = 0.623944
Epoch 150
Validation auc = 0.615493
Epoch 151
Validation auc = 0.617371
Epoch 152
Validation auc = 0.646009
Epoch 153
Validation auc = 0.684977
Epoch 154
Loss = 1.1033e+00, PNorm = 51.3103, GNorm = 29.2661, lr_0 = 7.7759e-04
Validation auc = 0.701408
Epoch 155
Validation auc = 0.705869
Epoch 156
Validation auc = 0.724413
Epoch 157
Validation auc = 0.729577
Epoch 158
Validation auc = 0.726291
Epoch 159
Loss = 1.3815e+00, PNorm = 51.3209, GNorm = 84.1671, lr_0 = 7.7095e-04
Validation auc = 0.707981
Epoch 160
Validation auc = 0.687089
Epoch 161
Validation auc = 0.670188
Epoch 162
Validation auc = 0.670188
Epoch 163
Validation auc = 0.671831
Epoch 164
Loss = 5.3251e-01, PNorm = 51.3301, GNorm = 56.3601, lr_0 = 7.6436e-04
Validation auc = 0.639906
Epoch 165
Validation auc = 0.622535
Epoch 166
Validation auc = 0.628638
Epoch 167
Validation auc = 0.652582
Epoch 168
Validation auc = 0.661972
Epoch 169
Loss = 1.0066e+00, PNorm = 51.3395, GNorm = 119.0251, lr_0 = 7.5782e-04
Validation auc = 0.656338
Epoch 170
Validation auc = 0.643662
Epoch 171
Validation auc = 0.639437
Epoch 172
Validation auc = 0.638967
Epoch 173
Validation auc = 0.639906
Epoch 174
Loss = 8.1192e-01, PNorm = 51.3489, GNorm = 53.3879, lr_0 = 7.5134e-04
Validation auc = 0.645070
Epoch 175
Validation auc = 0.645070
Epoch 176
Validation auc = 0.658216
Epoch 177
Validation auc = 0.660094
Epoch 178
Validation auc = 0.655869
Epoch 179
Loss = 2.9815e-01, PNorm = 51.3574, GNorm = 10.0070, lr_0 = 7.4492e-04
Validation auc = 0.658685
Epoch 180
Validation auc = 0.665258
Epoch 181
Validation auc = 0.664319
Epoch 182
Validation auc = 0.661502
Epoch 183
Validation auc = 0.673709
Epoch 184
Loss = 6.0346e-01, PNorm = 51.3657, GNorm = 58.8821, lr_0 = 7.3855e-04
Validation auc = 0.659624
Epoch 185
Validation auc = 0.661972
Epoch 186
Validation auc = 0.673709
Epoch 187
Validation auc = 0.682629
Epoch 188
Validation auc = 0.689671
Epoch 189
Loss = 6.9573e-01, PNorm = 51.3747, GNorm = 54.3049, lr_0 = 7.3224e-04
Validation auc = 0.692019
Epoch 190
Validation auc = 0.686385
Epoch 191
Validation auc = 0.688732
Epoch 192
Validation auc = 0.682160
Epoch 193
Validation auc = 0.679577
Epoch 194
Loss = 2.3704e-01, PNorm = 51.3830, GNorm = 18.0787, lr_0 = 7.2598e-04
Validation auc = 0.672300
Epoch 195
Validation auc = 0.668075
Epoch 196
Validation auc = 0.666667
Epoch 197
Validation auc = 0.656808
Epoch 198
Validation auc = 0.660094
Epoch 199
Loss = 1.0961e+00, PNorm = 51.3914, GNorm = 174.1376, lr_0 = 7.1977e-04
Validation auc = 0.646479
Epoch 200
Validation auc = 0.644601
Epoch 201
Validation auc = 0.639906
Epoch 202
Validation auc = 0.659624
Epoch 203
Validation auc = 0.662441
Epoch 204
Loss = 5.9541e-01, PNorm = 51.4010, GNorm = 33.3210, lr_0 = 7.1362e-04
Validation auc = 0.676056
Epoch 205
Validation auc = 0.677934
Epoch 206
Validation auc = 0.695775
Epoch 207
Validation auc = 0.702347
Epoch 208
Validation auc = 0.700939
Epoch 209
Loss = 4.5611e-01, PNorm = 51.4096, GNorm = 49.1670, lr_0 = 7.0752e-04
Validation auc = 0.688263
Epoch 210
Validation auc = 0.680282
Epoch 211
Validation auc = 0.680751
Epoch 212
Validation auc = 0.682160
Epoch 213
Validation auc = 0.676526
Epoch 214
Loss = 6.8738e-01, PNorm = 51.4173, GNorm = 77.9728, lr_0 = 7.0147e-04
Validation auc = 0.679343
Epoch 215
Validation auc = 0.676995
Epoch 216
Validation auc = 0.688732
Epoch 217
Validation auc = 0.691549
Epoch 218
Validation auc = 0.683099
Epoch 219
Loss = 4.5982e-01, PNorm = 51.4247, GNorm = 25.3253, lr_0 = 6.9548e-04
Validation auc = 0.682629
Epoch 220
Validation auc = 0.670892
Epoch 221
Validation auc = 0.673709
Epoch 222
Validation auc = 0.675117
Epoch 223
Validation auc = 0.670423
Epoch 224
Loss = 3.8800e-01, PNorm = 51.4321, GNorm = 40.7365, lr_0 = 6.8953e-04
Validation auc = 0.672770
Epoch 225
Validation auc = 0.661972
Epoch 226
Validation auc = 0.650235
Epoch 227
Validation auc = 0.644131
Epoch 228
Validation auc = 0.647418
Epoch 229
Loss = 9.0762e-01, PNorm = 51.4404, GNorm = 43.2434, lr_0 = 6.8364e-04
Validation auc = 0.663850
Epoch 230
Validation auc = 0.670423
Epoch 231
Validation auc = 0.665728
Epoch 232
Validation auc = 0.653991
Epoch 233
Validation auc = 0.643192
Epoch 234
Loss = 2.7495e-01, PNorm = 51.4486, GNorm = 13.1557, lr_0 = 6.7779e-04
Validation auc = 0.635211
Epoch 235
Validation auc = 0.634742
Epoch 236
Validation auc = 0.632394
Epoch 237
Validation auc = 0.630047
Epoch 238
Validation auc = 0.625352
Epoch 239
Loss = 3.3644e-01, PNorm = 51.4563, GNorm = 26.1510, lr_0 = 6.7200e-04
Validation auc = 0.621596
Epoch 240
Validation auc = 0.629108
Epoch 241
Validation auc = 0.640845
Epoch 242
Validation auc = 0.654460
Epoch 243
Validation auc = 0.658216
Epoch 244
Loss = 4.3877e-01, PNorm = 51.4639, GNorm = 13.7040, lr_0 = 6.6625e-04
Validation auc = 0.664319
Epoch 245
Validation auc = 0.666667
Epoch 246
Validation auc = 0.667136
Epoch 247
Validation auc = 0.666667
Epoch 248
Validation auc = 0.670423
Epoch 249
Loss = 5.4502e-01, PNorm = 51.4716, GNorm = 33.8288, lr_0 = 6.6056e-04
Validation auc = 0.669014
Epoch 250
Validation auc = 0.663380
Epoch 251
Validation auc = 0.672300
Epoch 252
Validation auc = 0.677700
Epoch 253
Validation auc = 0.679812
Epoch 254
Loss = 2.5118e-01, PNorm = 51.4801, GNorm = 18.0460, lr_0 = 6.5491e-04
Validation auc = 0.680282
Epoch 255
Validation auc = 0.677465
Epoch 256
Validation auc = 0.667136
Epoch 257
Validation auc = 0.663380
Epoch 258
Validation auc = 0.664789
Epoch 259
Loss = 2.2596e-01, PNorm = 51.4891, GNorm = 75.0728, lr_0 = 6.4931e-04
Validation auc = 0.666197
Epoch 260
Validation auc = 0.661972
Epoch 261
Validation auc = 0.651174
Epoch 262
Validation auc = 0.646948
Epoch 263
Validation auc = 0.659155
Epoch 264
Loss = 4.4055e-01, PNorm = 51.4975, GNorm = 29.2726, lr_0 = 6.4376e-04
Validation auc = 0.660563
Epoch 265
Validation auc = 0.665258
Epoch 266
Validation auc = 0.669014
Epoch 267
Validation auc = 0.669484
Epoch 268
Validation auc = 0.668545
Epoch 269
Loss = 2.9055e-01, PNorm = 51.5052, GNorm = 37.1446, lr_0 = 6.3826e-04
Validation auc = 0.666197
Epoch 270
Validation auc = 0.668075
Epoch 271
Validation auc = 0.676995
Epoch 272
Validation auc = 0.681690
Epoch 273
Validation auc = 0.684977
Epoch 274
Loss = 2.6131e-01, PNorm = 51.5118, GNorm = 20.9505, lr_0 = 6.3280e-04
Validation auc = 0.683099
Epoch 275
Validation auc = 0.683099
Epoch 276
Validation auc = 0.681221
Epoch 277
Validation auc = 0.685446
Epoch 278
Validation auc = 0.685915
Epoch 279
Loss = 3.0343e-01, PNorm = 51.5173, GNorm = 14.7723, lr_0 = 6.2739e-04
Validation auc = 0.689202
Epoch 280
Validation auc = 0.683099
Epoch 281
Validation auc = 0.679812
Epoch 282
Validation auc = 0.681221
Epoch 283
Validation auc = 0.683099
Epoch 284
Loss = 2.3389e-01, PNorm = 51.5217, GNorm = 24.2330, lr_0 = 6.2203e-04
Validation auc = 0.677934
Epoch 285
Validation auc = 0.676056
Epoch 286
Validation auc = 0.662441
Epoch 287
Validation auc = 0.652582
Epoch 288
Validation auc = 0.629108
Epoch 289
Loss = 1.5563e-01, PNorm = 51.5257, GNorm = 19.7276, lr_0 = 6.1671e-04
Validation auc = 0.620188
Epoch 290
Validation auc = 0.644601
Epoch 291
Validation auc = 0.653052
Epoch 292
Validation auc = 0.673239
Epoch 293
Validation auc = 0.682160
Epoch 294
Loss = 6.3875e-01, PNorm = 51.5303, GNorm = 36.4079, lr_0 = 6.1144e-04
Validation auc = 0.687793
Epoch 295
Validation auc = 0.689202
Epoch 296
Validation auc = 0.681221
Epoch 297
Validation auc = 0.675117
Epoch 298
Validation auc = 0.672770
Epoch 299
Loss = 2.0083e-01, PNorm = 51.5350, GNorm = 64.8785, lr_0 = 6.0621e-04
Validation auc = 0.682160
Model 0 best validation auc = 0.762911 on epoch 26
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.742324
Ensemble test auc = 0.742324
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=300, out_features=2000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2000, out_features=2000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.15000000000000002, inplace=False)
    (7): Linear(in_features=2000, out_features=1, bias=True)
  )
)
Number of parameters = 4,606,001
Moving model to cuda
Epoch 0
Warning: Found a task with predictions all 0s or all 1s
Validation auc = nan
Epoch 1
Validation auc = 0.683333
Epoch 2
Validation auc = 0.721127
Epoch 3
Validation auc = 0.599531
Epoch 4
Loss = 9.8391e+01, PNorm = 50.2707, GNorm = 2125.0354, lr_0 = 6.5000e-04
Validation auc = 0.682160
Epoch 5
Validation auc = 0.744601
Epoch 6
Validation auc = 0.642723
Epoch 7
Validation auc = 0.752113
Epoch 8
Validation auc = 0.625822
Epoch 9
Loss = 9.0534e+01, PNorm = 50.4059, GNorm = 1868.6551, lr_0 = 9.9743e-04
Validation auc = 0.545070
Epoch 10
Validation auc = 0.700469
Epoch 11
Validation auc = 0.709859
Epoch 12
Validation auc = 0.709390
Epoch 13
Validation auc = 0.711502
Epoch 14
Loss = 2.5807e+01, PNorm = 50.5600, GNorm = 1061.3010, lr_0 = 9.8890e-04
Validation auc = 0.717371
Epoch 15
Validation auc = 0.647418
Epoch 16
Validation auc = 0.525117
Epoch 17
Validation auc = 0.477230
Epoch 18
Validation auc = 0.699061
Epoch 19
Loss = 4.1106e+01, PNorm = 50.6676, GNorm = 1050.7429, lr_0 = 9.8045e-04
Validation auc = 0.715728
Epoch 20
Validation auc = 0.553521
Epoch 21
Validation auc = 0.733333
Epoch 22
Validation auc = 0.742723
Epoch 23
Validation auc = 0.688263
Epoch 24
Loss = 6.2797e+00, PNorm = 50.7421, GNorm = 162.5841, lr_0 = 9.7207e-04
Validation auc = 0.669484
Epoch 25
Validation auc = 0.688732
Epoch 26
Validation auc = 0.730986
Epoch 27
Validation auc = 0.724883
Epoch 28
Validation auc = 0.655399
Epoch 29
Loss = 1.1393e+01, PNorm = 50.7950, GNorm = 566.9681, lr_0 = 9.6376e-04
Validation auc = 0.565023
Epoch 30
Validation auc = 0.700000
Epoch 31
Validation auc = 0.708920
Epoch 32
Validation auc = 0.707042
Epoch 33
Validation auc = 0.648826
Epoch 34
Loss = 9.1311e+00, PNorm = 50.8338, GNorm = 400.9068, lr_0 = 9.5552e-04
Validation auc = 0.672770
Epoch 35
Validation auc = 0.629108
Epoch 36
Validation auc = 0.606808
Epoch 37
Validation auc = 0.660563
Epoch 38
Validation auc = 0.664319
Epoch 39
Loss = 6.0896e+00, PNorm = 50.8643, GNorm = 258.5170, lr_0 = 9.4735e-04
Validation auc = 0.672300
Epoch 40
Validation auc = 0.682864
Epoch 41
Validation auc = 0.705164
Epoch 42
Validation auc = 0.687793
Epoch 43
Validation auc = 0.678873
Epoch 44
Loss = 5.7510e+00, PNorm = 50.8927, GNorm = 191.9448, lr_0 = 9.3925e-04
Validation auc = 0.672770
Epoch 45
Validation auc = 0.645305
Epoch 46
Validation auc = 0.605399
Epoch 47
Validation auc = 0.596714
Epoch 48
Validation auc = 0.625352
Epoch 49
Loss = 3.5173e+00, PNorm = 50.9173, GNorm = 222.2598, lr_0 = 9.3122e-04
Validation auc = 0.649061
Epoch 50
Validation auc = 0.683568
Epoch 51
Validation auc = 0.677230
Epoch 52
Validation auc = 0.627465
Epoch 53
Validation auc = 0.652817
Epoch 54
Loss = 1.8330e+00, PNorm = 50.9396, GNorm = 88.1579, lr_0 = 9.2326e-04
Validation auc = 0.695775
Epoch 55
Validation auc = 0.670657
Epoch 56
Validation auc = 0.610798
Epoch 57
Validation auc = 0.588967
Epoch 58
Validation auc = 0.663850
Epoch 59
Loss = 2.0543e+00, PNorm = 50.9600, GNorm = 148.9299, lr_0 = 9.1537e-04
Validation auc = 0.678169
Epoch 60
Validation auc = 0.689671
Epoch 61
Validation auc = 0.675117
Epoch 62
Validation auc = 0.640141
Epoch 63
Validation auc = 0.615258
Epoch 64
Loss = 3.9227e+00, PNorm = 50.9799, GNorm = 226.9630, lr_0 = 9.0754e-04
Validation auc = 0.606808
Epoch 65
Validation auc = 0.605164
Epoch 66
Validation auc = 0.613850
Epoch 67
Validation auc = 0.582394
Epoch 68
Validation auc = 0.588732
Epoch 69
Loss = 4.1769e+00, PNorm = 51.0021, GNorm = 180.0101, lr_0 = 8.9978e-04
Validation auc = 0.665023
Epoch 70
Validation auc = 0.740610
Epoch 71
Validation auc = 0.732160
Epoch 72
Validation auc = 0.694836
Epoch 73
Validation auc = 0.672770
Epoch 74
Loss = 3.0201e+00, PNorm = 51.0249, GNorm = 132.3838, lr_0 = 8.9209e-04
Validation auc = 0.692019
Epoch 75
Validation auc = 0.725822
Epoch 76
Validation auc = 0.732864
Epoch 77
Validation auc = 0.715258
Epoch 78
Validation auc = 0.641315
Epoch 79
Loss = 3.0510e+00, PNorm = 51.0444, GNorm = 211.3002, lr_0 = 8.8447e-04
Validation auc = 0.682160
Epoch 80
Validation auc = 0.663380
Epoch 81
Validation auc = 0.706808
Epoch 82
Validation auc = 0.693897
Epoch 83
Validation auc = 0.684272
Epoch 84
Loss = 2.3472e+00, PNorm = 51.0640, GNorm = 72.6021, lr_0 = 8.7691e-04
Validation auc = 0.708920
Epoch 85
Validation auc = 0.698592
Epoch 86
Validation auc = 0.666667
Epoch 87
Validation auc = 0.631690
Epoch 88
Validation auc = 0.617371
Epoch 89
Loss = 1.2253e+00, PNorm = 51.0813, GNorm = 85.8997, lr_0 = 8.6941e-04
Validation auc = 0.612441
Epoch 90
Validation auc = 0.623474
Epoch 91
Validation auc = 0.596244
Epoch 92
Validation auc = 0.596948
Epoch 93
Validation auc = 0.620892
Epoch 94
Loss = 1.1676e+00, PNorm = 51.0972, GNorm = 117.0044, lr_0 = 8.6198e-04
Validation auc = 0.652582
Epoch 95
Validation auc = 0.673005
Epoch 96
Validation auc = 0.675117
Epoch 97
Validation auc = 0.699061
Epoch 98
Validation auc = 0.700235
Epoch 99
Loss = 1.0851e+00, PNorm = 51.1135, GNorm = 101.1609, lr_0 = 8.5461e-04
Validation auc = 0.691784
Epoch 100
Validation auc = 0.680282
Epoch 101
Validation auc = 0.667840
Epoch 102
Validation auc = 0.655399
Epoch 103
Validation auc = 0.670892
Epoch 104
Loss = 1.4115e+00, PNorm = 51.1282, GNorm = 118.7745, lr_0 = 8.4730e-04
Validation auc = 0.673005
Epoch 105
Validation auc = 0.668075
Epoch 106
Validation auc = 0.640610
Epoch 107
Validation auc = 0.640845
Epoch 108
Validation auc = 0.662441
Epoch 109
Loss = 1.0921e+00, PNorm = 51.1432, GNorm = 92.3640, lr_0 = 8.4006e-04
Validation auc = 0.666901
Epoch 110
Validation auc = 0.672066
Epoch 111
Validation auc = 0.658920
Epoch 112
Validation auc = 0.667136
Epoch 113
Validation auc = 0.663850
Epoch 114
Loss = 7.7813e-01, PNorm = 51.1576, GNorm = 79.6505, lr_0 = 8.3288e-04
Validation auc = 0.662676
Epoch 115
Validation auc = 0.667606
Epoch 116
Validation auc = 0.676995
Epoch 117
Validation auc = 0.680282
Epoch 118
Validation auc = 0.679343
Epoch 119
Loss = 8.2628e-01, PNorm = 51.1709, GNorm = 70.8590, lr_0 = 8.2576e-04
Validation auc = 0.665728
Epoch 120
Validation auc = 0.650235
Epoch 121
Validation auc = 0.646479
Epoch 122
Validation auc = 0.644601
Epoch 123
Validation auc = 0.651643
Epoch 124
Loss = 1.4127e+00, PNorm = 51.1848, GNorm = 41.2097, lr_0 = 8.1870e-04
Validation auc = 0.638028
Epoch 125
Validation auc = 0.629108
Epoch 126
Validation auc = 0.641549
Epoch 127
Validation auc = 0.655869
Epoch 128
Validation auc = 0.638028
Epoch 129
Loss = 1.2524e+00, PNorm = 51.1989, GNorm = 76.8009, lr_0 = 8.1170e-04
Validation auc = 0.619718
Epoch 130
Validation auc = 0.594836
Epoch 131
Validation auc = 0.593427
Epoch 132
Validation auc = 0.577465
Epoch 133
Validation auc = 0.561033
Epoch 134
Loss = 6.9649e-01, PNorm = 51.2130, GNorm = 65.6195, lr_0 = 8.0476e-04
Validation auc = 0.574648
Epoch 135
Validation auc = 0.604460
Epoch 136
Validation auc = 0.623005
Epoch 137
Validation auc = 0.639202
Epoch 138
Validation auc = 0.646244
Epoch 139
Loss = 5.0791e-01, PNorm = 51.2246, GNorm = 35.3866, lr_0 = 7.9788e-04
Validation auc = 0.633568
Epoch 140
Validation auc = 0.640376
Epoch 141
Validation auc = 0.673005
Epoch 142
Validation auc = 0.654930
Epoch 143
Validation auc = 0.632394
Epoch 144
Loss = 7.1566e-01, PNorm = 51.2358, GNorm = 47.6770, lr_0 = 7.9106e-04
Validation auc = 0.635211
Epoch 145
Validation auc = 0.626761
Epoch 146
Validation auc = 0.599061
Epoch 147
Validation auc = 0.598122
Epoch 148
Validation auc = 0.581690
Epoch 149
Loss = 1.1638e+00, PNorm = 51.2474, GNorm = 82.8641, lr_0 = 7.8430e-04
Validation auc = 0.564554
Epoch 150
Validation auc = 0.569953
Epoch 151
Validation auc = 0.567136
Epoch 152
Validation auc = 0.548826
Epoch 153
Validation auc = 0.545540
Epoch 154
Loss = 6.2217e-01, PNorm = 51.2591, GNorm = 46.7461, lr_0 = 7.7759e-04
Validation auc = 0.562911
Epoch 155
Validation auc = 0.611737
Epoch 156
Validation auc = 0.659155
Epoch 157
Validation auc = 0.662441
Epoch 158
Validation auc = 0.666197
Epoch 159
Loss = 7.4457e-01, PNorm = 51.2689, GNorm = 75.9286, lr_0 = 7.7095e-04
Validation auc = 0.671362
Epoch 160
Validation auc = 0.669014
Epoch 161
Validation auc = 0.654695
Epoch 162
Validation auc = 0.646714
Epoch 163
Validation auc = 0.640376
Epoch 164
Loss = 5.4168e-01, PNorm = 51.2774, GNorm = 47.1600, lr_0 = 7.6436e-04
Validation auc = 0.650704
Epoch 165
Validation auc = 0.647418
Epoch 166
Validation auc = 0.634272
Epoch 167
Validation auc = 0.632394
Epoch 168
Validation auc = 0.647887
Epoch 169
Loss = 6.3929e-01, PNorm = 51.2851, GNorm = 77.9567, lr_0 = 7.5782e-04
Validation auc = 0.640845
Epoch 170
Validation auc = 0.643662
Epoch 171
Validation auc = 0.656808
Epoch 172
Validation auc = 0.638967
Epoch 173
Validation auc = 0.614554
Epoch 174
Loss = 3.2387e-01, PNorm = 51.2943, GNorm = 17.6944, lr_0 = 7.5134e-04
Validation auc = 0.568545
Epoch 175
Validation auc = 0.526291
Epoch 176
Validation auc = 0.526761
Epoch 177
Validation auc = 0.553521
Epoch 178
Validation auc = 0.607512
Epoch 179
Loss = 8.2011e-01, PNorm = 51.3042, GNorm = 84.4027, lr_0 = 7.4492e-04
Validation auc = 0.642723
Epoch 180
Validation auc = 0.661033
Epoch 181
Validation auc = 0.669014
Epoch 182
Validation auc = 0.671831
Epoch 183
Validation auc = 0.666197
Epoch 184
Loss = 4.1261e-01, PNorm = 51.3133, GNorm = 50.2429, lr_0 = 7.3855e-04
Validation auc = 0.637089
Epoch 185
Validation auc = 0.633803
Epoch 186
Validation auc = 0.625822
Epoch 187
Validation auc = 0.603286
Epoch 188
Validation auc = 0.574648
Epoch 189
Loss = 2.9196e-01, PNorm = 51.3222, GNorm = 15.4661, lr_0 = 7.3224e-04
Validation auc = 0.573709
Epoch 190
Validation auc = 0.583099
Epoch 191
Validation auc = 0.591549
Epoch 192
Validation auc = 0.595305
Epoch 193
Validation auc = 0.607042
Epoch 194
Loss = 1.2117e-01, PNorm = 51.3327, GNorm = 23.2915, lr_0 = 7.2598e-04
Validation auc = 0.627700
Epoch 195
Validation auc = 0.636150
Epoch 196
Validation auc = 0.652113
Epoch 197
Validation auc = 0.662911
Epoch 198
Validation auc = 0.672770
Epoch 199
Loss = 4.2646e-01, PNorm = 51.3439, GNorm = 47.6482, lr_0 = 7.1977e-04
Validation auc = 0.678404
Epoch 200
Validation auc = 0.679812
Epoch 201
Validation auc = 0.676995
Epoch 202
Validation auc = 0.671362
Epoch 203
Validation auc = 0.670892
Epoch 204
Loss = 5.3703e-01, PNorm = 51.3534, GNorm = 30.0399, lr_0 = 7.1362e-04
Validation auc = 0.661972
Epoch 205
Validation auc = 0.652582
Epoch 206
Validation auc = 0.650235
Epoch 207
Validation auc = 0.653052
Epoch 208
Validation auc = 0.677465
Epoch 209
Loss = 5.5655e-01, PNorm = 51.3622, GNorm = 44.8964, lr_0 = 7.0752e-04
Validation auc = 0.686385
Epoch 210
Validation auc = 0.692958
Epoch 211
Validation auc = 0.690610
Epoch 212
Validation auc = 0.671831
Epoch 213
Validation auc = 0.639437
Epoch 214
Loss = 2.2465e-01, PNorm = 51.3711, GNorm = 35.0104, lr_0 = 7.0147e-04
Validation auc = 0.613615
Epoch 215
Validation auc = 0.601408
Epoch 216
Validation auc = 0.601408
Epoch 217
Validation auc = 0.605164
Epoch 218
Validation auc = 0.616432
Epoch 219
Loss = 7.9914e-01, PNorm = 51.3810, GNorm = 34.2654, lr_0 = 6.9548e-04
Validation auc = 0.629108
Epoch 220
Validation auc = 0.644601
Epoch 221
Validation auc = 0.655869
Epoch 222
Validation auc = 0.676995
Epoch 223
Validation auc = 0.691549
Epoch 224
Loss = 4.3360e-01, PNorm = 51.3900, GNorm = 14.5196, lr_0 = 6.8953e-04
Validation auc = 0.701408
Epoch 225
Validation auc = 0.696714
Epoch 226
Validation auc = 0.697183
Epoch 227
Validation auc = 0.696244
Epoch 228
Validation auc = 0.671362
Epoch 229
Loss = 2.4616e-01, PNorm = 51.3977, GNorm = 17.0608, lr_0 = 6.8364e-04
Validation auc = 0.647887
Epoch 230
Validation auc = 0.632864
Epoch 231
Validation auc = 0.637559
Epoch 232
Validation auc = 0.636620
Epoch 233
Validation auc = 0.621596
Epoch 234
Loss = 2.3837e-01, PNorm = 51.4041, GNorm = 20.3016, lr_0 = 6.7779e-04
Validation auc = 0.604695
Epoch 235
Validation auc = 0.598592
Epoch 236
Validation auc = 0.611737
Epoch 237
Validation auc = 0.622535
Epoch 238
Validation auc = 0.631455
Epoch 239
Loss = 2.6026e-01, PNorm = 51.4104, GNorm = 37.5413, lr_0 = 6.7200e-04
Validation auc = 0.646948
Epoch 240
Validation auc = 0.648826
Epoch 241
Validation auc = 0.652113
Epoch 242
Validation auc = 0.652582
Epoch 243
Validation auc = 0.661502
Epoch 244
Loss = 3.2490e-01, PNorm = 51.4171, GNorm = 20.6622, lr_0 = 6.6625e-04
Validation auc = 0.669014
Epoch 245
Validation auc = 0.665728
Epoch 246
Validation auc = 0.651174
Epoch 247
Validation auc = 0.648357
Epoch 248
Validation auc = 0.643662
Epoch 249
Loss = 6.3574e-01, PNorm = 51.4239, GNorm = 23.9548, lr_0 = 6.6056e-04
Validation auc = 0.643662
Epoch 250
Validation auc = 0.622066
Epoch 251
Validation auc = 0.625822
Epoch 252
Validation auc = 0.648357
Epoch 253
Validation auc = 0.673239
Epoch 254
Loss = 3.5639e-01, PNorm = 51.4306, GNorm = 60.9870, lr_0 = 6.5491e-04
Validation auc = 0.678404
Epoch 255
Validation auc = 0.668545
Epoch 256
Validation auc = 0.656338
Epoch 257
Validation auc = 0.640845
Epoch 258
Validation auc = 0.641315
Epoch 259
Loss = 4.9918e-01, PNorm = 51.4377, GNorm = 34.6150, lr_0 = 6.4931e-04
Validation auc = 0.639437
Epoch 260
Validation auc = 0.644131
Epoch 261
Validation auc = 0.627230
Epoch 262
Validation auc = 0.620657
Epoch 263
Validation auc = 0.619249
Epoch 264
Loss = 3.2870e-01, PNorm = 51.4453, GNorm = 41.0447, lr_0 = 6.4376e-04
Validation auc = 0.621596
Epoch 265
Validation auc = 0.626291
Epoch 266
Validation auc = 0.621127
Epoch 267
Validation auc = 0.614085
Epoch 268
Validation auc = 0.616432
Epoch 269
Loss = 3.0420e-01, PNorm = 51.4516, GNorm = 20.8948, lr_0 = 6.3826e-04
Validation auc = 0.610798
Epoch 270
Validation auc = 0.603756
Epoch 271
Validation auc = 0.602347
Epoch 272
Validation auc = 0.603286
Epoch 273
Validation auc = 0.594836
Epoch 274
Loss = 2.2746e-01, PNorm = 51.4578, GNorm = 29.8066, lr_0 = 6.3280e-04
Validation auc = 0.591549
Epoch 275
Validation auc = 0.599061
Epoch 276
Validation auc = 0.602817
Epoch 277
Validation auc = 0.610798
Epoch 278
Validation auc = 0.621596
Epoch 279
Loss = 1.6913e-01, PNorm = 51.4639, GNorm = 21.6851, lr_0 = 6.2739e-04
Validation auc = 0.636150
Epoch 280
Validation auc = 0.644601
Epoch 281
Validation auc = 0.643662
Epoch 282
Validation auc = 0.630047
Epoch 283
Validation auc = 0.593897
Epoch 284
Loss = 3.4355e-01, PNorm = 51.4705, GNorm = 41.9486, lr_0 = 6.2203e-04
Validation auc = 0.591080
Epoch 285
Validation auc = 0.606573
Epoch 286
Validation auc = 0.614085
Epoch 287
Validation auc = 0.617840
Epoch 288
Validation auc = 0.620657
Epoch 289
Loss = 2.1029e-01, PNorm = 51.4774, GNorm = 18.3590, lr_0 = 6.1671e-04
Validation auc = 0.628169
Epoch 290
Validation auc = 0.640845
Epoch 291
Validation auc = 0.655399
Epoch 292
Validation auc = 0.652582
Epoch 293
Validation auc = 0.643662
Epoch 294
Loss = 3.6815e-01, PNorm = 51.4838, GNorm = 38.3337, lr_0 = 6.1144e-04
Validation auc = 0.641315
Epoch 295
Validation auc = 0.655399
Epoch 296
Validation auc = 0.664789
Epoch 297
Validation auc = 0.669014
Epoch 298
Validation auc = 0.678404
Epoch 299
Loss = 1.6380e-01, PNorm = 51.4898, GNorm = 20.1322, lr_0 = 6.0621e-04
Validation auc = 0.682160
Model 0 best validation auc = 0.752113 on epoch 7
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.603251
Ensemble test auc = 0.603251
10-fold cross validation
	Seed 0 ==> test auc = 0.670078
	Seed 1 ==> test auc = 0.707706
	Seed 2 ==> test auc = 0.664058
	Seed 3 ==> test auc = 0.729681
	Seed 4 ==> test auc = 0.723359
	Seed 5 ==> test auc = 0.732390
	Seed 6 ==> test auc = 0.732691
	Seed 7 ==> test auc = 0.840458
	Seed 8 ==> test auc = 0.742324
	Seed 9 ==> test auc = 0.603251
Overall test auc = 0.714600 +/- 0.058770
Elapsed time = 0:03:08
